Single-node alert evaluation for high availability

Grafana alerting has added single-node alert evaluation for high availability clusters. When enabled, only one Grafana instance in the HA cluster evaluates alert rules, reducing query load on data sources.

Normally, when you run multiple instances of Grafana, all alert rules are evaluated on all instances by default. With single-node evaluation enabled, the Grafana cluster automatically chooses a primary instance that is responsible for evaluating all alert rules. Other instances skip evaluation entirely.

With single-node evaluation, the primary instance broadcasts fired alerts to all other instances through the cluster communication channel. This ensures that every instance’s embedded Alertmanager has the current alerts, which is necessary for failure recovery and for the Alertmanager API to return correct data on all instances.

If the primary instance becomes unavailable, the cluster reassigns positions and a new instance becomes responsible for evaluation. During failure recovery, there is a brief gap in evaluations. Existing alert states remain in the database.

With single-node evaluation, you can reduce datasource query load dramatically while keeping HA benefits for notifications.