Legacy Grafana alerts
Grafana 8.0 has new and improved alerts. The new alerting system are an opt-in feature that centralizes alerting information for Grafana managed alerts and alerts from Prometheus-compatible data sources in one UI and API.
Out of the box, Grafana still supports legacy dashboard alerts. Legacy Grafana alerts consists of two parts:
Alert rules - When the alert is triggered. Alert rules are defined by one or more conditions that are regularly evaluated by Grafana. Notification channel - How the alert is delivered. When the conditions of an alert rule are met, the Grafana notifies the channels configured for that alert.
Currently only the graph panel visualization supports alerts.
Legacy aerts have two main components:
- Alert rule - When the alert is triggered. Alert rules are defined by one or more conditions that are regularly evaluated by Grafana.
- Notification channel - How the alert is delivered. When the conditions of an alert rule are met, the Grafana notifies the channels configured for that alert.
You can perform the following tasks for alerts:
- Create an alert rule
- View existing alert rules and their current state
- Test alert rules and troubleshoot
- Add or edit an alert contact point
Currently alerting supports a limited form of high availability. Since v4.2.0 of Grafana, alert notifications are deduped when running multiple servers. This means all alerts are executed on every server but no duplicate alert notifications are sent due to the deduping logic. Proper load balancing of alerts will be introduced in the future.
Grafana managed alerts are evaluated by the Grafana backend. Rule evaluations are scheduled, according to the alert rule configuration, and queries are evaluated by an engine that is part of core Grafana.
Alert rules can only query backend data sources with alerting enabled:
- builtin or developed and maintained by grafana:
Google Cloud Monitoring,
Azure Data Explorer
- any community backend data sources with alerting enabled (
alertingproperties are set in the plugin.json)
Metrics from the alert engine
The alert engine publishes some internal metrics about itself. You can read more about how Grafana publishes internal metrics.
||gauge||How many alerts by state|
||histogram||Histogram of requests to the Alerting API|
||gauge||The number of active, non default alertmanager configurations for grafana managed alerts|
||counter||The total number of rule evaluations|
||counter||The total number of rule evaluation failures|
||summary||The duration for a rule to execute|
||gauge||The number of rules|
Related Grafana resources
GrafanaCONline 2021 is coming June 7-17
Be the first to learn about exciting next-generation features in Grafana 8.0, be inspired by what community members are building, and attend expert-led sessions and workshops on Grafana, Prometheus, Loki logs, and more.
Unify your data with Grafana plugins: Splunk, MongoDB, Datadog, and more
Show how Grafana can be used to take data from multiple different sources and unify it, without disrupting the investments that are working today.
Demo: Getting started with Grafana Enterprise and observability
Join the Grafana Labs team for a 30-minute demo of how to get started with the Grafana Stack, so you can go from zero to observability in just a few minutes.