Documentation Grafana Cloud Observe and act Manage on-call and incidents Get started

Get started with Grafana IRM

Set up Grafana IRM so that when an alert fires, IRM notifies the person on call and they can acknowledge it. This page walks you through each piece in order, shows you how to test the result, and points you to where you can go deeper. For how alerts, schedules, and escalation fit together, refer to Introduction to Grafana IRM.

If someone else set things up and you’ve just been added to IRM, refer to Configure your user profile instead.

Before you begin

To follow this page, you need the following:

• A Grafana Cloud stack with access to IRM. If you don’t have a Grafana Cloud account, sign up for free today.
• The Admin role in Grafana. The Editor role can’t create integrations or escalation chains. For details, refer to Manage access for Grafana IRM.

You don’t need teammates or a team to start. You can set everything up for yourself and add people later. If you already have teams, you can assign a team to each schedule and escalation chain when you create it, or later. For details about teams and user roles, refer to Access and teams.

If you’re migrating from another on-call tool or managing IRM as code, start with Migrate to Grafana IRM or Infrastructure as code instead.

Your setup path

Set up IRM in this order:

1. Create an on-call schedule.
2. Create an escalation chain that notifies the schedule.
3. Set up how you get notified.
4. Connect Grafana Alerting to IRM.
5. Send a demo alert to test the setup.

The demo alert is how you test this setup. After it works, point a Grafana Alerting alert rule at the contact point IRM created. Refer to Connect a real alert rule.

If you prefer a guided version of the same setup, follow the learning path:

Start your learning experience with Grafana Learning Paths

Grafana Learning Paths provide a clear, structured path that leads you from beginner concepts to advanced use cases. Learn about this Grafana feature on Configure Grafana IRM for incident response.

Start learning

Create an on-call schedule

Escalation chains notify whoever is on call right now, so start with one rotation and add yourself first. A test alert notifies you, not your teammates.

To create a schedule:

1. In the Grafana Cloud main menu, go to Alerts & IRM > IRM > Schedules.
2. Click + New schedule.
3. On Set up on-call rotation schedule, click Create.
4. Enter a name for the schedule, and click Create schedule.
5. When the rotation settings open, under Add user, add yourself, and click Create.

You can add teammates to the rotation now or later. The first user in the list is on call first.

For more information, refer to Create on-call schedules, Import schedules, and Shift swaps and overrides.

Create an escalation chain

An escalation chain decides who is notified, when, and what happens if nobody acknowledges. Build one chain that notifies your schedule, waits, and then notifies the next person on call.

To create an escalation chain:

1. In IRM, go to Escalation chains, and click New escalation chain.
2. Enter a name for the chain, and click Create escalation chain.
3. From Add escalation step, add these steps in order:
   1. Notify users from on-call schedule, and select your schedule.
   2. Wait, and select 5 minutes.
   3. Notify next on-call shift from schedule, and select the same schedule.

If you’re the only person on call, the third step notifies you again. After teammates join the rotation, it reaches whoever is on call next.

To route different alerts to different chains, group related alerts, or change how alerts look, refer to Configure alert escalation and routing in Grafana IRM.

For more information about escalation chains, refer to Configure escalation chains in Grafana IRM.

Set up how you get notified

IRM reaches you through your own notification rules. If you have no rules set up, IRM sends notification to the email associated with your Grafana Cloud account. Connect the Grafana mobile app for push notifications, or verify a phone number for SMS and phone calls.

To set up your notifications:

1. Click your user avatar, and select Profile.
2. To use push notifications, click Mobile app, connect the Grafana mobile app, and click Send test push.
3. To use SMS or phone calls, click IRM settings, then Phone verification. Verify your phone number, and click Send test SMS or Make test call.
4. On IRM settings, click Notification rules. Add the rules you want under Default notification rules and Important notification rules.

For more information, refer to Configure notification rules, Connect notification channels, Phone and SMS notifications, Get started with the Grafana mobile app, and Configure your user profile.

Connect Grafana Alerting to IRM

Grafana Alerting sends alerts into IRM through a contact point. When you add the Grafana Alerting integration, IRM creates that contact point for you. Then you set your escalation chain on the integration’s default route. Without that step, nobody is notified, because a new integration’s default route has no escalation chain.

To connect Grafana Alerting:

1. In IRM, go to Integrations, and on Monitoring systems, click Add integration.
2. Select Grafana Alerting. It shows Quick connect.
3. Enter a name for the integration.
4. Under Grafana Alerting contact point, click Create a new one, and enter a name for the contact point.
5. Click Create integration. IRM opens the integration’s page.
6. On the Default route, in Trigger escalation chain, select your escalation chain.

To receive alerts from another monitoring tool, you can add its integration instead.

For more information, refer to Grafana Alerting integration for Grafana IRM, Install and manage integrations for Grafana IRM, and Alert source integrations for Grafana IRM.

Send a demo alert

A demo alert tests your setup without an alert rule. Each demo alert creates its own alert group and notifies real people, which is why you put yourself on call first.

To send a demo alert:

1. On the integration’s page, click Send demo alert, and then click Send alert.
2. In IRM, go to Alert groups, and open the new alert group.
3. In the timeline, watch the escalation chain run until the notification reaches you.
4. Click Acknowledge. Escalation stops.
5. After you finish investigating, click Resolve.

After you acknowledge the alert group, new alerts that join it don’t trigger a new escalation.

For more information, refer to Respond to alerts in Grafana IRM.

Result

Your setup works when the demo alert produces all of the following:

• A new alert group on the Alert groups page.
• A timeline that shows which escalation chain ran and who was on call.
• Notification attempts in the timeline, and a notification you received.
• Your acknowledgment, which stopped escalation.

Connect a real alert rule

So far, the demo alert is the only alert that has reached your integration. A real alert reaches IRM when a Grafana Alerting alert rule sends notifications to the contact point you created. IRM creates the contact point, but no alert rule uses it yet.

In Grafana Alerting, the contact point has the name you gave it and the Grafana IRM type. To create or update the alert rule, refer to Configure alert rules.

Next steps

After the demo alert works, practice incident response and get your teammates ready.

Manage incidents

Practice by declaring a drill incident. In IRM, go to Incidents, click Declare, and then click Declare drill. Adding responders to a drill notifies real people. Drills don’t appear in incident insights. If you’ve connected Jira or GitHub, a drill can create linked issues with titles that start with [DRILL].

Connect Slack to give each incident its own channel, or Microsoft Teams to give each incident a conversation thread. Then customize how your team responds to incidents.

For more information, refer to Declare a drill incident, Configure Slack for Grafana IRM, Microsoft Teams integration for Grafana IRM, and Customize incident response process.

Prepare your responders

When you invite people, review and assign each person’s user role, so they have the permissions for how they’ll use IRM and the tasks you expect of them. Responders need the Editor role or the OnCaller role.

For details, refer to What each role can do.

After you invite them, each teammate sets their own notification rules and installs the Grafana mobile app. Until they set rules, IRM only emails them.

Refer to Respond to alerts in Grafana IRM and Shift swaps and overrides for how to acknowledge an alert and swap a shift.

For more information, refer to Configure notification rules and Grafana mobile app.

Go further

When you’re ready to expand your setup, start with these pages:

• To send different alerts to different escalation chains, refer to Configure routing rules in Grafana IRM.
• To control which alerts group together, refer to Configure alert grouping.
• To change how alerts look and behave, refer to Customize alert templates in Grafana IRM.
• To connect more monitoring, chat, and development tools, refer to Connect integrations in Grafana IRM.
• To track how your response is working, refer to Measure and improve incident response in Grafana IRM.
• Refer to Best practices for Grafana IRM.