Configure alert rules on Grafana Labs

Configure Grafana-managed alert rules

Mon, 16 Mar 2026 22:19:55 +0000

Configure Grafana-managed alert rules

Grafana-managed rules can query data from multiple data sources in a single alert rule. They are the most flexible alert rule type. You can also add expressions to transform your data, set alert conditions, and images in alert notifications.

Note
In Grafana Cloud, the number of Grafana-managed alert rules you can create depends on your Grafana Cloud plan.

Free Forever plan: You can create up to 100 free alert rules, with each alert rule having a maximum of 1000 alert instances.

All paid plans (Pro and Advanced): They have a soft limit of 2000 alert rules and support unlimited alert instances. To increase the limit, open a support ticket from the Cloud portal.

To create or edit Grafana-managed alert rules, follow the instructions below. For a practical example, check out our tutorial on getting started with Grafana alerting.

Before you begin

Verify that the data sources you plan to query in the alert rule are compatible with Grafana-managed alert rules and are properly configured.

Permissions

Only users with Edit permissions for the folder storing the rules can edit or delete Grafana-managed alert rules.

Provisioning

Note that if you delete an alert resource created in the UI, you can no longer retrieve it.

To backup and manage alert rules, you can provision alerting resources using options such as configuration files, Terraform, or the Alerting API.

Default vs Advanced options

You can use default or advanced options for Grafana-managed alert rule creation. The default options streamline rule creation with a cleaner header and a single query and condition. For more complex rules, use advanced options to add multiple queries and expressions.

You can toggle between the two options. Once you have created an alert rule, the system defaults to your previous choice for the next alert rule.

Switching from advanced to default may result in queries and expressions that cannot be converted. In this case, a warning message asks if you want to continue to reset to default settings.

Default and advanced options are enabled by default for Grafana Cloud users and this feature is being rolled out progressively. OSS users can enable them via the alertingQueryAndExpressionsStepMode feature toggle.

Set alert rule name

Click Alerts & IRM -> Alert rules -> + New alert rule.
Enter a name to identify your alert rule.

This name is displayed in the alert rule list. It is also the alertname label for every alert instance that is created from this rule.

Define query and condition

Define a query to get the data you want to measure and a condition that needs to be met before an alert rule fires.

You can toggle between Default and Advanced options. If the Default vs. Advanced feature is not enabled in your Grafana instance, follow the Advanced options instructions.

Set folder and labels

Organize your alert rule with a folder and set of labels.

In the Labels section, you can optionally choose whether to add labels to organize your alert rules and their notifications. For more details, refer to alert rule labels.

Select a folder or click + New folder.
Add labels, if required.

Add custom labels by selecting existing key-value pairs from the drop down, or add new labels by entering the new key or value.

Configure alert evaluation behavior

Use alert rule evaluation to determine how frequently an alert rule should be evaluated and how quickly it should change its state.

To do this, you need to make sure that your alert rule is in the right evaluation group and set a pending period time that works best for your use case.

Select a folder or click + New folder.
Select an evaluation group or click + New evaluation group.

If you are creating a new evaluation group, specify the interval for the group.

All rules within the same group are evaluated concurrently over the same time interval.
Enter a pending period.

The pending period is the period in which an alert rule can be in breach of the condition until it fires.

Once a condition is met, the alert goes into the Pending state. If the condition remains active for the duration specified, the alert transitions to the Firing state, else it reverts to the Normal state.
Turn on pause alert notifications, if required.

You can pause alert rule evaluation to prevent noisy alerting while tuning your alerts. Pausing stops alert rule evaluation and doesn’t create any alert instances. This is different to mute timings, which stop notifications from being delivered, but still allows for alert rule evaluation and the creation of alert instances.

In Configure no data and error handling, you can define the alerting behavior and alerting state for two scenarios:

When the evaluation returns No data or all values are null.
When the evaluation returns Error or timeout.

Configure no data and error handling

Configure	Set alert state	Description
No Data	No Data	The default option for No Data events. Sets alert instance state to `No Data`. The alert rule also creates a new alert instance `DatasourceNoData` with the name and UID of the alert rule, and UID of the datasource that returned no data as labels.
Error	Error	The default option for Error events. Sets alert instance state to `Error`. The alert rule also creates a new alert instance `DatasourceError` with the name and UID of the alert rule, and UID of the datasource that returned no data as labels.
No Data or Error	Alerting	Sets the alert instance state to `Pending` and then transitions to `Alerting` once the pending period ends. If you sent the pending period to 0, the alert instance state is immediately set to `Alerting`.
No Data or Error	Normal	Sets alert instance state to `Normal`.
No Data or Error	Keep Last State	Maintains the alert instance in its last state. Useful for mitigating temporary issues.

For more details, refer to alert instance states and modify the no data or error state.

Configure labels and notifications

In the Labels section, you can optionally choose whether to add labels to organize your alert rules, make searching easier, as well as set which notification policy should handle your firing alert instance.

In the Configure notifications section, you can choose to select a contact point directly from the alert rule form or choose to use notification policy routing as well as set up mute timings and groupings.

Complete the following steps to set up labels and notifications.

Add labels, if required.

Add custom labels by selecting existing key-value pairs from the drop down, or add new labels by entering the new key or value.
Configure who receives a notification when an alert rule fires by either choosing Select contact point or Use notification policy.

Select contact point
1. Choose this option to select an existing contact point.
  
  All notifications for this alert rule are sent to this contact point automatically and notification policies are not used.
2. You can also optionally select a mute timing as well as groupings and timings to define when not to send notifications.
  
  Note
  An auto-generated notification policy is generated. Only admins can view these auto-generated policies from the Notification policies list view. Any changes have to be made in the alert rules form.
Use notification policy
1. Choose this option to use the notification policy tree to direct your notifications.
  
  Note
  All alert rules and instances, irrespective of their labels, match the default notification policy. If there are no nested policies, or no nested policies match the labels in the alert rule or alert instance, then the default notification policy is the matching policy.
2. Preview your alert instance routing set up.
  
  Based on the labels added, alert instances are routed to the following notification policies displayed.
3. Expand each notification policy below to view more details.
4. Click See details to view alert routing details and an email preview.

Configure notification message

Use annotations to add information to alert messages that can help respond to the alert.

Annotations are included by default in notification messages, and can use text or templates to display dynamic data from queries.

Grafana provides several optional annotations.

Optional: Add a summary.

Short summary of what happened and why.
Optional: Add a description.

Description of what the alert rule does.
Optional: Add a Runbook URL.

Webpage where you keep your runbook for the alert
Optional: Add a custom annotation.

Add any additional information that could help address the alert.
Optional: Link dashboard and panel.

Link the alert rule to a panel to facilitate alert investigation.
Click Save rule.

Configure data source-managed alert rules

Mon, 16 Mar 2026 22:19:55 +0000

Configure data source-managed alert rules

Data source-managed alert rules can only be created using Grafana Mimir or Grafana Loki data sources.

The rules are stored within the data source. In a distributed architecture, they can scale horizontally to provide high-availability. For more details, refer to alert rule types.

We recommend using Grafana-managed alert rules whenever possible and opting for data source-managed alert rules when scaling your alerting setup is necessary.

Rules from a Prometheus data source appear in the Data source-managed section of the Alert rules page when Manage alerts via Alerting UI is enabled.

However, Grafana can only create and edit data source-managed rules for Mimir and Loki, not for a Prometheus instance.

To create or edit data source-managed alert rules, follow these instructions.

Before you begin

Verify that you have write permission to the Mimir or Loki data source. Otherwise, you cannot create or update data source-managed alert rules.

Enable the Ruler API

For more information, refer to the Mimir Ruler API or Loki Ruler API.

Mimir - use the /prometheus prefix. The Prometheus data source supports both Grafana Mimir and Prometheus, and Grafana expects that both the Query API and Ruler API are under the same URL. You cannot provide a separate URL for the Ruler API.
Loki - The local rule storage type, default for the Loki data source, supports only viewing of rules. To edit rules, configure one of the other rule storage types.

Permissions

Alert rules for Mimir or Loki instances can be edited or deleted by users with Editor or Admin roles.

If you do not want to manage alert rules for a particular data source, go to its settings and clear the Manage alerts via Alerting UI checkbox.

Provisioning

Note that if you delete an alert resource created in the UI, you can no longer retrieve it.

To backup and manage alert rules, you can provision alerting resources using options such as configuration files, Terraform, or the Alerting API.

Set alert rule name

Click Alerts & IRM -> Alert rules -> + New alert rule.
Enter a name to identify your alert rule.

This name is displayed in the alert rule list. It is also the alertname label for every alert instance that is created from this rule.

Define query and condition

Define a query to get the data you want to measure and a condition that needs to be met before an alert rule fires.

Note
By default, new alert rules are Grafana-managed. To switch to Data source-managed, follow these instructions.

Select a Prometheus-based data source from the drop-down list.

You can also click Open advanced data source picker to find more options.
Enter a PromQL or LogQL query, including the alert condition.
In the Rule type option, select Data source-managed.
Click Preview alerts.

Set alert evaluation behavior

Use alert rule evaluation to determine how frequently an alert rule should be evaluated and how quickly it should change its state.

Select a namespace or click + New namespace.
Select an evaluation group or click + New evaluation group.

If you are creating a new evaluation group, specify the interval for the group.

All rules within the same group are evaluated sequentially over the same time interval. You can reorder them from the Alert rules page.
Enter a pending period.

The pending period is the period in which an alert rule can be in breach of the condition until it fires.

Once a condition is met, the alert goes into the Pending state. If the condition remains active for the duration specified, the alert transitions to the Firing state, else it reverts to the Normal state.

Configure labels and notifications

Add labels to your alert rules to set which notification policy should handle your firing alert instances.

All alert rules and instances, irrespective of their labels, match the default notification policy. If there are no nested policies, or no nested policies match the labels in the alert rule or alert instance, then the default notification policy is the matching policy.

Add labels if you want to change the way your notifications are routed.

Add custom labels by selecting existing key-value pairs from the drop down, or add new labels by entering the new key or value.

Configure notification message

Use annotations to add information to alert messages that can help respond to the alert.

Annotations are included by default in notification messages, and can use text or templates to display dynamic data from queries.

Grafana provides several optional annotations.

Optional: Add a summary.

Short summary of what happened and why.
Optional: Add a description.

Description of what the alert rule does.
Optional: Add a Runbook URL.

Webpage where you keep your runbook for the alert
Optional: Add a custom annotation.

Add any additional information that could help address the alert.
Optional: Link dashboard and panel.

Link the alert rule to a panel to facilitate alert investigation.
Click Save rule.

Create and link alert rules to panels

Mon, 16 Mar 2026 22:19:55 +0000

Create and link alert rules to panels

Grafana allows you to link an alert rule to a dashboard panel. This can help you:

Inform alert responders about where to investigate and which data to examine.
Visualize the alert state directly from dashboards.

An alert rule is linked to a panel by setting the dashboardUId and panelId annotations. Both annotations must be set together.

Link alert rules to panels

When configuring the alert rule, you can set the dashboard and panel annotations as shown in this video.

Configure the alert rule.
In the Configure notification message section, click Link dashboard and panel.
Select an existing dashboard, then choose a panel from the selected dashboard.
Complete the alert rule configuration and click Save rule to initiate the alert rule.

You can then view the alert state on the panel.

Create alert rules from panels

To streamline alert creation, you can create an alert rule directly from a panel.

Navigate to a dashboard in the Dashboards section.
Hover over the top-right corner of a panel and click the panel menu icon.
From the dropdown menu, select More… > New alert rule.
This opens the Edit rule form and pre-fills some values:
- Sets the dashboardUId and panelId annotations to the corresponding dashboard and panel.
- Sets the alert rule query using the panel query.
Complete the alert rule configuration and click Save rule to initiate the alert rule.

You can then view the alert state on the panel.

Note
Changes to panel and alert rule queries aren’t synchronized. If you change a query, you have to update it in both the panel and the alert rule.

Access linked alert rules from panels

This option is available only in time series panels. To access alert rules associated to a time series panel, complete the following steps.

Hover over the top-right corner of the panel and click the panel menu icon.
Click Edit.
Click the Alert tab to view existing alert rules or create a new one.

Create recording rules

Mon, 16 Mar 2026 22:19:55 +0000

Configure recording rules

Recording rules allows you to periodically pre-compute frequently used or computationally expensive queries, saving the results as a new time series metric.

For instance, you can create a recording rule generating a new metric, error_9001_count, which counts occurrences of a specific log error within one minute. Then, query the error_9001_count metric in dashboards and alert rules.

Recording rules can be helpful in various scenarios, such as:

Faster queries are needed: Performing heavy aggregations or querying large data sets is quicker with precomputed results than real-time queries.
Reducing system load: Precomputing specific queries in advance can reduce system overload caused by multiple simultaneous queries.
Simplifying complex aggregations: Create a new metric from complex aggregations to facilitate alert and dashboard setup.
Reusing queries across alerts: Improve efficiency by reusing the same query across similar alert rules and dashboards.

The evaluation group of the recording rule determines how often the metric is pre-computed.

Similar to alert rules, Grafana supports two types of recording rules:

Grafana-managed recording rules, which can query any Grafana data source supported by alerting.
Data source-managed recording rules, which can query Prometheus-based data sources like Mimir or Loki.

Template annotations and labels

Mon, 16 Mar 2026 22:19:55 +0000

Template annotations and labels

You can use templates to customize alert and notification messages, including dynamic data from alert rule queries.

In Grafana Alerting, you can template alert messages in two ways.

Template annotations and labels: In the alert rule definition, you can template annotations and labels to include extra information from query data to the alert, adding meaningful details based on the query results.
Template notifications: You can template notifications to control the content and appearance of your notifications.

How templating works

In this diagram, you can see the differences between both types of templates.

Refer to Templates Introduction for a more detailed explanation of this diagram.

Both types of templates are written in the Go templating system. However, it’s important to understand that variables and functions used in notification templates are different from those used in annotation and label templates.

Template annotations and labels: These templates add extra information to individual alert instances. Template variables like $labels and $values represent alert query data of the individual alert instance.
Template notifications: Notification templates format the notification content for a group of alerts. Variables like .Alerts include all firing and resolved alerts in the notification.

Template annotations

Annotations add additional information to alert instances and are often used to help identify the alert and guide responders on how to address the issue.

Annotations are key-value pairs defined in the alert rule. They can contain plain text or template code that is evaluated when the alert fires.

Grafana includes several optional annotations, such as description, summary, runbook_url, dashboardUId and panelId, which can be edited in the alert rule. You can also create your custom annotations. For example, you might create a new annotation named location to report the location of the system that triggered the alert.

Here’s an example of a summary annotation explaining why the alert was triggered, using plain text.

CPU usage has exceeded 80% for the last 5 minutes.

However, if you want to display dynamic query values in annotations, you need to use template code. Common use cases include:

Displaying the query value that triggered the alert.
Highlighting label information that identifies the alert, such as the environment, instance, or region.
Providing specific instructions based on query values.
Customizing runbook links depending on query labels.
Including contact information based on query labels.

For instance, you can template the previous example to display the specific instance and CPU value that triggered the alert.

CPU usage for {{ $labels.instance }} has exceeded 80% ({{ $values.A.Value }}) for the last 5 minutes.

Alternatively, you can use the index function to print query values.

CPU usage for {{ index $labels "instance" }} has exceeded 80% ({{ index $values "A" }}) for the last 5 minutes.

The result of the annotation would be as follows.

CPU usage for Instance 1 has exceeded 80% (81.2345) for the last 5 minutes.

How to template an annotation

Complete the following steps to template an annotation.

Navigate to Alerts & IRM -> Alert rules -> create or edit an alert rule.
Scroll down to the Configure notification message section.
Copy in your template in the corresponding annotation field (summary, description, runbook_url, custom).

Preview annotation templates

You can template annotations when creating or editing an alert rule.

Two common methods are used to test and preview annotation templates:

Trigger the alert and view the alert instance state in the Grafana UI, where all annotations of the alert instance are displayed.
Use a notification template that displays all annotations, then preview the notification template using the alert instance.

Template labels

The set of labels for an alert instance is used to uniquely identify that alert among all other alert instances.

Labels determine how alerts are routed and managed for notifications, making their design key to the effectiveness of your alerting system.

Labels can be returned from an alert rule query, such as the pod label in a Kubernetes Prometheus query. You can also define additional labels in the alert rule to provide extra information for processing alerts.

Like annotations, labels are key-value pairs that can contain plain text or template code evaluated when the alert fires.

Template labels when the labels returned by your queries are insufficient. For instance:

A new label based on a query value can group a subset of alerts differently, changing how notifications are sent.
A new label based on a query value can be used in a notification policy to alter the notification contact point.

Here’s an example of templating a severity label based on the query value.

{{ if (gt $values.A.Value 90.0) -}}
critical
{{ else if (gt $values.A.Value 80.0) -}}
high
{{ else if (gt $values.A.Value 60.0) -}}
medium
{{ else -}}
low
{{- end }}

In this example, the value of the severity label is determined by the query value, and the possible options are critical, high, medium, or low. You can then use the severity label to change their notifications—for instance, sending critical alerts immediately or routing low alerts to a specific team for further review.

Note
You should avoid displaying query values in labels, as this may create numerous unique alert instances—one for each distinct label value. Instead, use annotations for query values.

How to template a label

Complete the following steps to template a label.

Navigate to Alerts & IRM -> Alert rules -> create or edit an alert rule.
Scroll down to the Configure labels and notifications section.
Click + Add labels.
Enter a key that identifies the label.
Copy in your template in the value field.

Preview label templates

You can template label values when creating or editing an alert rule.

To preview label values, select Use notification policy, and then click on Preview routing.

More information

For further details on how to template alert rules, refer to:

Tip
For a practical example of templating, refer to our Getting Started with Templating tutorial.