About alerting high availability
The Grafana Alerting system has two main components: a
Scheduler and an internal
Scheduler evaluates your alert rules, while the internal Alertmanager manages routing and grouping.
When running Grafana Alerting in high availability, the operational mode of the scheduler remains unaffected, and each Grafana instance evaluates all alerts. The operational change happens in the Alertmanager when it deduplicates alert notifications across Grafana instances.
The coordination between Grafana instances happens via a Gossip protocol. Alerts are not gossiped between instances and each scheduler delivers the same volume of alerts to each Alertmanager.
The two types of messages gossiped between Grafana instances are:
- Notification logs: Who (which instance) notified what (which alert).
- Silences: If an alert should fire or not.
The notification logs and silences are persisted in the database periodically and during a graceful Grafana shut down.
For configuration instructions, refer to enable alerting high availability.
Related Grafana resources
Unify your data with Grafana plugins: Splunk, MongoDB, Datadog, and more
Show how Grafana can be used to take data from multiple different sources and unify it, without disrupting the investments that are working today.
Getting started with Grafana Enterprise and observability
Join the Grafana Labs team for a 30-minute demo of how to get started with the Grafana Stack, so you can go from zero to observability in just a few minutes.