SLI examples for Grafana SLO on Grafana Labs

SLI example for availability

Sat, 04 Apr 2026 20:10:38 +0000

SLI example for availability

This guide provides examples to show how to define availability SLIs based on successful HTTP responses and probe results. The examples explain various methods to define SLIs using distinct Prometheus metric types.

First, it’s important to understand what Grafana SLO expects from an SLI.

Grafana supports event-based SLIs, which measure the ratio of successful events to total events.
The SLI query result must return a ratio between 0 and 1, where 1 means 100% of events were successful.
Grafana expects SLIs to be expressed as a ratio-like query: numerator / denominator.

Depending on the type of metric, you can calculate the ratio of successful events using counts or rates:

Success ratio = number of successful events (over a period)
                /
                total number of events (over a period)

Success ratio = rate of successful events (over a period)
                /
                rate of total events (over a period)

Both formulas produce the ratio of successful events, a value between 0 and 1.

In the Grafana SLO wizard, you can create these SLIs using two methods:

Ratio query builder: Enter a counter metric for success events and a counter metric for total events, and it auto-generates the final SLI query.
Advanced: Enter the ratio SLI query directly.

HTTP availability (using Prometheus Counter)

HTTP availability is a common SLI for frontend and API services, defining availability as requests that do not return server errors (5xx status codes).

Number of successful events: All non-5xx requests
Total number of events: All HTTP requests

This example uses http_requests_total, a Prometheus counter metric that counts the number of HTTP requests by status code, method, and other labels.

You can get both the number of successful events and the total number of events from this metric, and define the SLI by selecting the Ratio option in the Grafana SLO wizard:

Ratio query builder	Value	Description
Success metric	`http_requests_total{status!~"5.."}`	Metric for success requests
Total metric	`http_requests_total`	Metric for total requests
Grouping	(leave empty)	Creates a single SLI dimension See the multidimensional SLI example

Click Run queries to generate the final SLI ratio query:

(
  sum(rate(http_requests_total{status!~"5.."}[$__rate_interval] offset 2m))
  or 0 * sum(rate(http_requests_total[$__rate_interval] offset 2m))
)
/
sum(rate(http_requests_total[$__rate_interval] offset 2m))

Breakdown of the ratio SLI query

The auto-generated SLI query includes several common elements for building reliable SLIs and uses the ratio of successful event rates formula:

Success ratio = rate of successful events (over a period)
                /
                rate of total events (over a period)

The SLI query result is a ratio between 0 and 1, where 1 means 100% of events were successful.

Rate of successful events

The following part of the query measures the rate of successful events over the rate interval:

(
  sum(rate(http_requests_total{status!~"5.."}[$__rate_interval] offset 2m))
  or 0 * sum(rate(http_requests_total[$__rate_interval] offset 2m))
)

http_requests_total{status!~"5.."}: Returns only successful requests.
rate(...[$__rate_interval]): Calculates the per-second rate of successful requests over the recommended rate interval.
offset 2m: Shifts the query two minutes into the past to account for scrape or data ingestion delays.
sum(...): Aggregates across all series (dimensions) to get the total success rate.
or 0 * sum(rate(http_requests_total...)): Fallback for missing data. If the success metric returns no data, it returns 0 when dividing by total events.

This numerator is then divided by the rate of total requests.

Rate of total events

/
sum(rate(http_requests_total[$__rate_interval] offset 2m))

This query follows the rate formula and applies the same offset for alignment with the numerator (rate of successful events).

The full query returns a ratio between 0 and 1, representing the proportion of successful requests. The Grafana SLO wizard displays the final SLI query and a graph of its results:

Note that you can also use the Advanced SLI option to create the same SLI query directly.

Probe availability (using Prometheus Summary)

This example uses Synthetic Monitoring probes, such as local probes or Grafana Cloud probes, to verify service availability.

The process is as follows:

Configure a synthetic check from one or more probe locations that continuously verify system availability.

The check runs regularly, based on the configured frequency, and stores its results in Prometheus.
Define an SLO whose SLI queries the Prometheus probe results.

Grafana SLO then evaluates the SLI query and reports the SLO compliance.

This example uses the probe_all_success metric, a summary metric that tracks whether the probe succeeded.

Prometheus summary metrics expose additional *_sum and *_count series. You can use these to get the number of successful events and the total number of events, and define the SLI using the Ratio option in the Grafana SLO wizard:

Ratio query builder	Value	Description
Success metric	`probe_all_success_sum{job="<JOB_NAME>"}`	Number of successful probes
Total metric	`probe_all_success_count{job="<JOB_NAME>"}`	Total number of probe executions
Grouping	(leave empty)	Creates a single SLI dimension See the multidimensional SLI example

Click Run queries to generate the final SLI ratio query:

(
  sum(rate(probe_all_success_sum{job="<JOB_NAME>"}[$__rate_interval] offset 2m))
  or 0 * sum(rate(probe_all_success_count{job="<JOB_NAME>"}[$__rate_interval] offset 2m))
)
/
sum(rate(probe_all_success_count{job="<JOB_NAME>"}[$__rate_interval] offset 2m))

The SLI query returns a ratio between 0 and 1, where 1 means 100% of probe executions were successful.

This example works exactly like the HTTP availability example. To learn why the SLI is formed this way and how it works, refer to the breakdown of the ratio SLI query .

Probe availability (using Prometheus Gauge)

Sometimes, a binary gauge metric is used to track successes, such as the probe_success metric.

probe_success is 1 on success.
probe_success is 0 on failure.

In the SLO wizard, the Ratio option expects a counter metric and cannot generate the correct ratio SLI for this case. Use the Advanced query option instead.

Define an SLI that returns the ratio of successes, represented as a value between 0 and 1, as in the previous examples. You can use the ratio of successful events formula for this SLI:

Success ratio = number of successful events (over a period)
                /
                total number of events (over a period)

The SLI can then be defined as follows:

# `sum_over_time` sums the 1s to calculate the number of successful probes
sum(sum_over_time(probe_success{job="<JOB_NAME>"}[$__interval]))
/
sum(count_over_time(probe_success{job="<JOB_NAME>"}[$__interval]))
# `count_over_time` counts the total number of probe executions (1=success, 0=failure)

probe_success{job="<JOB_NAME>"}: Returns probe results for the specified job. Each sample is either 1 (success) or 0 (failure).
sum_over_time(...[$__interval]): Sums the values of successful probes over the given interval; failed probes are counted as 0.
count_over_time(...[$__interval]): Counts all probe executions in the given interval, including both successes (1) and failures (0).
sum(...): Aggregates across all series (dimensions) to get the total number of successful probes and total probe executions.

Like the other SLI examples, this SLI returns a value between 0 and 1, representing the ratio of successful executions.

Note
$__rate_interval is recommended for calculating rate() in other examples. When you use other _over_time() functions that don’t require at least two data points, it’s better to use $__interval to achieve finer error budget resolution by evaluating SLIs at smaller time intervals.

Availability using failure metric

Sometimes, you might have a metric that counts failures instead of successes, because your instrumentation doesn’t use a single metric with a label indicating success or failure.

For example, you may only have the following counters:

failure_total: Counts failed requests or operations.
all_total: Counts all requests or operations.

In this case, you can calculate availability by subtracting failures from the total:

Success ratio = (total events - failed events)
                /
                total events

Avoid using 1 - (<failure rate> / <total rate>) because the SLI can’t be parsed as a ratio-like query.

In the Grafana SLO wizard, use the Advanced option:

# rate of successful events
(
  sum(rate(all_total[$__rate_interval]))
  -
  sum(rate(failure_total[$__rate_interval]))
)
/
sum(rate(all_total[$__rate_interval]))
# rate of total events

The SLI query result is still the same: a ratio between 0 and 1, where 1 means 100% of events were successful.

SLI example for latency

Sat, 04 Apr 2026 20:10:38 +0000

SLI example for latency

This guide provides examples of how to define latency SLIs using different Prometheus metric types. The basic SLO example for demonstration purposes is as follows:

SLI category	SLI description	Target	Time window
Latency	Requests respond within 2 seconds	99%	28d

The SLI in this example includes all requests, and the SLO defines the target percentage.

When possible, avoid using percentiles in SLIs, such as 95th percentile latency with a 99% target, to maintain simplicity and consistency across SLO types. Refer to Building good SLOs—CRE life lessons from Google Cloud for more on this topic.

Before you begin, read the SLI availability examples to understand how SLIs are defined in Grafana SLO:

Note
Grafana expects SLIs to parse as a ratio-like query: numerator / denominator.

The SLI query result must return a ratio between 0 and 1, where 1 means 100% of events were successful. This is required to evaluate whether the SLI meets the SLO target.

Probe latency (using Prometheus Gauge)

This example uses the probe_duration_seconds metric from Synthetic Monitoring probes to verify public latency. For details on how Synthetic Monitoring probes work, see the SLI availability examples using probes.

Metric	Type	Description
`probe_duration_seconds`	Gauge	How long the probe took to complete in seconds

In the Grafana SLO wizard, you can create SLIs using two options:

Ratio query builder: Enter counter metrics for successful and total events.
Advanced: Enter the ratio SLI query directly.

Because probe_duration_seconds is not a counter metric, choose the Advanced option to create the SLI query.

SLIs are defined as ratio-like queries, either as the ratio of successful events or the ratio of successful event rates:

# ratio of successful event rates formula
Success ratio = rate of successful events (over a period)
                /  
                rate of total events (over a period)

# ratio of successful events formula
Success ratio = number of successful events (over a period)
                /  
                total number of events (over a period)

With gauge metrics, you can implement the ratio of successful events formula as follows:

# number of successful probe requests over the interval
sum(
  count_over_time(
    (probe_duration_seconds{job="<JOB_NAME>"} < 2)[$__interval:]
  )
)
/
# number of total probe requests over the interval
sum(
  count_over_time(
    probe_duration_seconds{job="<JOB_NAME>"}[$__interval:]
  )
)

Here’s the breakdown of the numerator query:

# number of successful probe requests over the interval
sum(
  count_over_time(
    (probe_duration_seconds{job="<JOB_NAME>"} < 2)[$__interval:]
  )
)

probe_duration_seconds{job="<JOB_NAME>"} < 2

Returns probe latency samples. The < 2 comparison filters samples where latency is within the SLI threshold (less than two seconds).

The result is a binary series: 1 for success and no sample for failure.
[$__interval:]

Runs the previous expression over the past $__interval.

Because count_over_time works only on range vectors, it uses a subquery [:] to produce a range vector containing all samples from that period.
count_over_time(...) Counts the number of samples in the previous query, the number of successful probe requests in the range vector.
Finally, sum(...) aggregates across all series (dimensions).

The numerator is then divided by the total number of probe requests over the same interval using a similar query:

/
# number of total probe requests over the interval
sum(
  count_over_time(
    probe_duration_seconds{job="<JOB_NAME>"}[$__interval:]
  )
)

Note
$__rate_interval is recommended for calculating rate() in other examples. When you use other _over_time() functions that don’t require at least two data points, it’s better to use $__interval to achieve finer error budget resolution by evaluating SLIs at smaller time intervals.

Alternatively, the numerator can use bool and sum_over_time:

# number of successful probe requests over the interval
# `bool` returns a binary 0/1 series and `sum_over_time` sums 1s for successes
sum(
    sum_over_time(
      (probe_duration_seconds{job="<JOB_NAME>"} < bool 2)[$__interval:]
    )
)
/
# number of total probe requests over the interval
sum(
  count_over_time(
    probe_duration_seconds{job="<JOB_NAME>"}[$__interval:]
  )
)

Probe latency (using Classic Histogram)

The SLI example uses the probe_all_duration_seconds histogram metric, whose SLI query is different.

Metric	Type	Description
`probe_all_duration_seconds`	Histogram	How long the probe took to complete in seconds

Classic histogram metrics in Prometheus store samples based on their value (latency in this case) and expose additional series:

*_count: Returns all samples for all latencies.
*_bucket: Returns samples per configured buckets. The buckets for this metric are 0, 0.005, 0.1, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, and +Inf.

You can use a histogram metric to return the number of successful samples if the metric includes a bucket for the specific SLI threshold.

However, probe_all_duration_seconds does not include a bucket for 2s, and cannot be used to filter histogram samples at that threshold. For alternatives, refer to handle a threshold not available as a bucket.

This example uses a different threshold (2.5s) for demonstration purpose. Use the Ratio option to build the SLI query as follows:

Ratio query builder	Value	Description
Success metric	`probe_all_duration_seconds_bucket{job="<JOB_NAME>", le="2.5"}`	Number of probes requests under 2.5s
Total metric	`probe_all_duration_seconds_count{job="<JOB_NAME>"}`	Total number of probe requests
Grouping	(leave empty)	Creates a single SLI dimension See the multidimensional SLI example

Click Run queries to generate the final SLI ratio query:

The auto-generated SLI implements the ratio of successful event rates formula:

Success ratio = rate of successful events (over a period)
                /  
                rate of total events (over a period)

The SLI query returns a ratio between 0 and 1, where 1 means 100% of events were successful.

To learn why the auto-generated SLI is formed this way and how it works, refer to the breakdown of the ratio SLI query of the HTTP availability example.

Handle a threshold not available as a bucket

With classic histograms, it’s common for your SLI threshold to not match an existing histogram bucket, as in this example:

The SLI searches for responses under 2 seconds.
But the available buckets are configured for 1 and 2.5, not 2.

In this case, probe_all_duration_seconds_bucket{job="<JOB_NAME>", le="2"} does not work, and you should consider other approaches:

Add a bucket for your threshold: If you control the instrumentation, update the histogram metric to include a bucket for the exact SLI threshold.
Use a fallback metric: Check if a latency metric is available like in the previous Gauge example, or the native histogram example below.
Approximate using the nearest bucket: Use the nearest higher or lower bucket. Document this clearly and adapt your SLO settings, as the SLO no longer match the intended SLI threshold.

HTTP latency (using Native Histogram)

In contrast to classic histograms, Prometheus native histograms can estimate the number of observations within arbitrary ranges—not just preconfigured buckets.

histogram_fraction() returns the estimated fraction of observations within a specified range, as a value between 0 and 1.

This function must be used with rate() or increase() to correctly handle counter resets. The SLI query should follow the ratio of successful event rates formula:

# ratio of successful event rates formula
Success ratio = rate of successful events (over a period)
                /  
                rate of total events (over a period)

This example uses http_request_duration_seconds, a native histogram metric that records request latency. The complete SLI formula look like this:

(
histogram_fraction(0, 2, sum(rate(http_request_duration_seconds[$__rate_interval])))
*
histogram_count(         sum(rate(http_request_duration_seconds[$__rate_interval])))
)
/
histogram_count(         sum(rate(http_request_duration_seconds[$__rate_interval])))

rate(http_request_duration_seconds[$__rate_interval]) calculates the per-second rate of observations over the interval.
sum (...) aggregates across all dimensions (for example, all endpoints or status codes).
histogram_fraction(0, 2, ...) estimates the fraction of the request rate with latency between 0 and 2 seconds during the interval.
histogram_count(...) returns the total request rate for the interval.

The query follows the ratio-style format (numerator/denominator) of event-based SLIs:

Numerator: histogram_fraction(...) * histogram_count(...).
Denominator: histogram_count(...).
It returns a ratio between 0 and 1, where 1 means 100% of requests were under 2 seconds.

Note
For consistency, this example uses histogram_count(sum()).

However, sum(histogram_count()) returns the same query result and is often more efficient.

Multidimensional SLI example

Sat, 04 Apr 2026 20:10:38 +0000

Multidimensional SLI example

This guide explains what multidimensional SLIs are, how they work, and provides a basic example.

Before you begin, read the SLI availability examples to understand how SLIs are defined in Grafana SLO:

Grafana supports event-based SLIs, which measure the ratio of successful events to total events.
The SLI query result must return a ratio between 0 and 1, where 1 means 100% of events were successful.
Grafana expects SLIs to be expressed as a ratio-like query: numerator / denominator.

In the availability and latency SLI examples, some SLIs are calculated using the ratio of successful event rates formula:

Success ratio = rate of successful events over a period
                /  
                rate of total events over a period

However, this formula is not fully accurate, because Prometheus queries can return multiple series (also called dimensions).

In practice, the final SLIs use sum(...) to aggregate results from all series. The formula looks more like:

Success ratio = sum(rate of successful events over a period)
                /  
                sum(rate of total events over a period)

Here, sum(...) aggregates all potential dimensions (all distinct label values) in the numerator and denominator before the final ratio calculation.

This type of SLI is referred to as a roll-up SLI (or aggregated SLI). The following is an example using the Ratio query builder:

Multidimensional probe example

Multidimensional SLIs (SLIs evaluated across multiple label dimensions) use sum by (<labels>) in both the numerator and denominator, producing multiple ratio series. For example:

Success ratio = sum by (probe) (rate of successful probe executions)
                /  
                sum by (probe) (rate of total probe executions)

Dimension	Success ratio per dimension
{probe=“NorthVirginia”}	0.9
{probe=“Spain”}	1
{probe=“Tokyo”}	0.95

The final SLI result is 0.9, equal to its equivalent roll-up SLI.

In the final SLI calculation, all dimensions are aggregated, making it act as a roll-up SLI for SLO compliance.

Continuing with the previous example, use the Grouping function to define dimensions per probe:

Note that you can create multidimensional SLIs using either option in the Grafana SLO wizard: Ratio or Advanced.

How multidimensional SLIs work

For SLO compliance evaluation, the SLI calculation for multidimensional and roll-up SLIs is exactly the same.

However, Grafana SLO provides additional functionality to handle multidimensional SLIs:

Fast and slow burn alerts per dimension. When enabled, Grafana SLO triggers fast-burn or slow-burn alerts whenever an individual dimension consumes the error budget quickly or slowly, respectively.

The SLO dashboard displays the list of multidimensional alerts

Note
Multidimensional alerts are not triggered for overall SLO consumption, but only for the consumption of a particular dimension.

To be alerted when the overall error budget is consumed, create a roll-up SLO that does not include the sum/group by dimensions.
Per-dimension SLO dashboard filtering. The SLO dashboard allows filtering results and visualizing SLI consumption for each dimension.

Tip
To learn more about how Grafana evaluates metrics with multiple label dimensions, see also Multi-dimensional alert examples.