--- title: "Grafana Alerting integration for Grafana IRM | Grafana Cloud documentation" description: "Understand how Grafana Alerting and Grafana IRM work together, including grouping, resolving, silences, and routing." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Grafana Alerting integration for Grafana IRM This page explains the concepts behind connecting Grafana Alerting to Grafana IRM in Grafana Cloud: how alerts move between the two products, and how grouping, resolving, silences, and routing behave. For setup and configuration steps, refer to [Connect Grafana Alerting and Grafana IRM](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/integrations/grafana-alerting/connect). ## How Grafana Alerting works with Grafana IRM Grafana Alerting and Grafana IRM work together to page on-call responders when your alert rules fire. Alerting decides when to send notifications, how to group alerts, and when they resolve. IRM decides who gets paged and how. ``` --- config: fontFamily: "Inter, Helvetica, Arial, sans-serif" themeVariables: lineColor: "#8b9099" fontSize: "14px" flowchart: nodeSpacing: 28 rankSpacing: 40 padding: 14 subGraphTitleMargin: top: 6 bottom: 10 --- flowchart LR accTitle: How an alert moves from Grafana Alerting to Grafana IRM accDescr { In Grafana Alerting, an alert rule fires, a notification policy matches its labels and applies Group by, and the Grafana IRM contact point sends one notification per group. In Grafana IRM, the Grafana Alerting integration receives it, the first matching route picks the alert group, and the escalation chain pages responders. You configure the notification policy and the escalation chain yourself. } subgraph alerting["Grafana Alerting"] direction TB rule["`**Alert rule** fires alerts`"] --> policy["`**Notification policy** matches labels, applies Group by`"] policy --> cp["`**Grafana IRM contact point** one notification per group`"] end subgraph irm["Grafana IRM"] direction TB int["`**Grafana Alerting integration** receives the notification`"] --> route["`**Route** first match wins`"] route --> ag["`**Alert group** new or open`"] ag --> ec["`**Escalation chain** pages responders`"] end alerting --> irm classDef step fill:#ffffff,stroke:#F05A28,stroke-width:1px,rx:6,ry:6,color:#1f2937 classDef configure fill:#ffffff,stroke:#F05A28,stroke-width:2px,stroke-dasharray:5 4,rx:6,ry:6,color:#1f2937 classDef stage fill:#f6f7f9,stroke:#e4e6eb,stroke-width:1px,color:#5b6170 class rule,cp,int,route,ag step class policy,ec configure class alerting,irm stage ``` You configure the two dashed steps yourself: a notification policy in Alerting and an escalation chain in IRM. The contact point sends one notification for each Alerting notification group, not for each alert. ## The life of an alert group Here’s what happens to one alert group, from the first firing notification to the resolve: ``` --- config: fontFamily: "Inter, Helvetica, Arial, sans-serif" themeVariables: fontSize: "14px" lineColor: "#8b9099" actorBkg: "#ffffff" actorBorder: "#F05A28" actorTextColor: "#1f2937" actorLineColor: "#d0d3d9" signalColor: "#8b9099" signalTextColor: "#1f2937" sequenceNumberColor: "#ffffff" sequence: mirrorActors: true --- sequenceDiagram accTitle: The life of an alert group accDescr { Grafana Alerting sends a firing notification for a new group, and IRM creates an alert group and pages the responder. Repeat notifications for the same group join the open alert group. The responder acknowledges in IRM. When every alert in the group clears, Alerting sends a resolved notification and the alert group resolves. } autonumber box rgb(246, 247, 249) participant A as Grafana Alerting participant I as Grafana IRM participant R as Responder end A->>I: Firing, new group I->>R: New alert group, responder paged A->>I: Repeat, same group R->>I: Acknowledge A->>I: Resolved, every alert cleared I-->>R: Alert group resolves ``` IRM adds repeat notifications to the open alert group without restarting the escalation. Two cases work differently: Expand table | When this happens in Grafana Alerting | You see this in Grafana IRM | |-----------------------------------------------|-------------------------------------------------------------------------------------| | Some alerts in the notification group resolve | The alert group stays firing until all of them clear | | The alert is silenced or muted | Alerting sends nothing. If the alert resolves meanwhile, the alert group stays open | Responders acknowledge alert groups in IRM. Acknowledging or resolving an alert group in IRM doesn’t change the alert rule in Alerting. ## Grouping The **Group by** setting in Alerting controls how alerts are grouped in IRM. Notifications from the same Alerting notification group join the same open IRM alert group. Alerts that match different IRM routes never share an alert group. Repeat notifications don’t restart the escalation, but they do count toward escalation steps that check how many alerts arrive in a time window. For example, suppose a `HighLatency` rule fires three alerts: Expand table | Alert | `service` | `instance` | |-------|------------|------------| | 1 | `checkout` | `a` | | 2 | `checkout` | `b` | | 3 | `cart` | `c` | Here’s how many IRM alert groups you get with two **Group by** settings: Expand table | **Group by** | Notification groups | IRM alert groups | |------------------------|------------------------------------------------------------------|------------------| | `alertname` | One group with all three alerts | 1 | | `alertname`, `service` | One group for `checkout` (alerts 1, 2), one for `cart` (alert 3) | 2 | To get one alert group per team or service, add that label to **Group by**. These changes in Alerting start new IRM alert groups for new alerts: - Changing the notification policy tree or its matchers - Changing **Group by** - Renaming a contact point that’s selected directly in an alert rule If you make one of these changes while alerts are firing, the old alert group stays open. Resolve it in IRM. > Caution > > Don’t edit the **Grouping Id** template on the integration. IRM uses it to match resolved notifications to the right alert group. If you change it, alert groups stop resolving automatically. Change grouping with **Group by** in Alerting instead. For more information, refer to [Group alert notifications](https://grafana.com/docs/grafana-cloud/observe-and-act/alert-and-measure-reliability/alerting/fundamentals/notifications/group-alert-notifications/) and [Configure alert grouping](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/escalation-and-routing/alert-grouping). ## Resolving An IRM alert group resolves automatically when every alert in its Alerting notification group has resolved. Until then, it stays firing. After it resolves, the next firing notification starts a new alert group. The resolved notification must match the same IRM route as the firing notifications. If it matches a different route, the original alert group stays open. > Note > > Keep **Disable resolved message** turned off on the **Grafana IRM** contact point. If you turn it on, Alerting doesn’t send resolved notifications, and IRM alert groups stay firing until someone resolves them. Alerting also resolves alerts whose series stop returning data, after two evaluations by default. IRM receives these as resolved notifications. Refer to [Stale alert instances](https://grafana.com/docs/grafana-cloud/observe-and-act/alert-and-measure-reliability/alerting/fundamentals/alert-rule-evaluation/stale-alert-instances/). To resolve alert groups automatically when an escalation finishes, add the **Resolve alert group automatically** step to the escalation chain. ## Silences and maintenance Alerting and IRM each manage their own silences, so you can quiet alerts at the level that fits the situation. An IRM silence pauses paging for one alert group and leaves Alerting as it is. If the alert resolves during an IRM silence, IRM resolves the alert group when the silence ends. An Alerting silence stops notifications at the source and leaves existing IRM alert groups as they are. Expand table | Situation | Use | What happens | |---------------------------------------------------|---------------------------------------------------|----------------------------------------------------------------------| | Stop paging for one alert group you’re working on | Silence the alert group in IRM | Escalation pauses, and Alerting keeps sending notifications | | Planned work on the systems behind an integration | IRM maintenance mode | Alerting isn’t affected, and the effect in IRM depends on the mode | | Stop notifications for matching alerts | A silence in Alerting | Alerting sends nothing to IRM, including resolves during the silence | | Recurring quiet hours | A mute timing or active time interval in Alerting | Alerting sends nothing to IRM, including resolves while muted | > Warning > > If an alert resolves while it’s silenced or muted in Alerting, IRM never receives the resolve, and the alert group stays firing. After an Alerting silence or mute timing ends, check IRM for alert groups that are still firing and resolve them. Silencing one alert in a notification group doesn’t resolve it in IRM. Later notifications leave that alert out. IRM maintenance mode offers **Silence escalations**, **Group alerts**, and **Disable alerts**. Refer to [Manage maintenance mode](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/integrations/configure-integrations/#manage-maintenance-mode), [Configure silences](https://grafana.com/docs/grafana-cloud/observe-and-act/alert-and-measure-reliability/alerting/configure-notifications/create-silence/), and [Configure mute timings and active time intervals](https://grafana.com/docs/grafana-cloud/observe-and-act/alert-and-measure-reliability/alerting/configure-notifications/mute-timings/). ## How routing works across both products Routing happens in two stages. Notification policies in Alerting choose which alerts reach IRM, for example, only critical and warning alerts. IRM routes then choose which escalation chain runs, for example, by team. IRM checks routes from top to bottom and uses the first match. The default route comes last and catches everything else. Unlike notification policies, IRM routes don’t nest. To write route templates for Alerting notifications, refer to [Route alerts by team and severity](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/integrations/grafana-alerting/connect/#route-alerts-by-team-and-severity). ## Next steps - [Connect Grafana Alerting and Grafana IRM](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/integrations/grafana-alerting/connect) - [Configure alert grouping](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/escalation-and-routing/alert-grouping) - [Configure routing rules](https://grafana.com/docs/grafana-cloud/observe-and-act/respond-to-incidents/escalation-and-routing/routing-rules) - [Group alert notifications](https://grafana.com/docs/grafana-cloud/observe-and-act/alert-and-measure-reliability/alerting/fundamentals/notifications/group-alert-notifications/)