Documentation Grafana Cloud Alerts and IRM IRM Configure Integrations Configure Slack

Configure Slack for Grafana IRM

The Grafana Cloud app for Slack brings your incident response and on-call workflows directly into your Slack workspace, helping your team respond to alerts and manage incidents with less friction.

You can configure Slack notifications for alert escalations, set personal notification preferences, receive on-call shift updates, declare and manage incidents, and interact with IRM functionality using slash commands and emoji reactions.

Before you begin

Before configuring Slack for IRM, you must complete the following:

• Install the Grafana Cloud app for Slack
• Connect your Slack user profile to your Grafana Cloud account

To install the Grafana Cloud app for Slack, refer to Connect your Slack workspace to Grafana Cloud.

After installation, you can configure the IRM-specific Slack settings and features described in this document.

For information about required permissions, refer to the Permissions and Scope documentation.

Connect your Slack user to Grafana IRM

To use the /grafana slash command and receive personal Slack notifications from IRM, you must link your Slack user account to your Grafana Cloud account.

To link your accounts:

1. In Grafana Cloud, click your avatar and go to your Profile.
2. Select the IRM tab.
3. In the Slack connection section, click Open Slack connection page.
4. Authorize your Slack account and follow the prompts to verify and link your Slack account.

After linking your Slack account, you can use IRM functionality in Slack according to the role and permissions assigned to you in Grafana. The actions you can take in Slack depend on your permissions in Grafana IRM.

For more information on roles and permissions, refer to the Grafana Cloud user roles and permissions documentation.

Disconnect your Slack user from IRM

If you need to unlink your Slack user from a Grafana Cloud stack, you can do so from the same tab in your Grafana user profile. Grafana Cloud Admins can also manage linked Slack profiles on behalf of users in their organization. For more information, refer to Manage linked Slack profiles.

Configure alert notifications

There are two ways you can configure Slack notifications for alerts, with escalation chains and personal notification rules.

You can use escalation chains to route alerts to Slack channels or user groups as an escalation step. Whereas, personal notification rules allow individual users to define how they are notified of alerts, including Slack mentions.

Configure escalation chains with Slack notifications

Escalation chains define how alerts are routed and escalated when they fire. You can add Slack notification steps to escalation chains to notify your team in Slack channels or by mentioning Slack user groups.

There are two Slack notification methods available in escalation chains:

• Notify all Slack channel members: Posts a notification to a Slack channel which notifies all members of that channel. This option sends a message that’s visible to everyone in the channel and follows Slack’s default notification behavior for all channel members. Use this method when you want to alert an entire team or broadcast alerts to a shared channel.

• Notify Slack user group: Mentions a specific Slack user group in a notification message. This option directs notifications to a subset of users based on user group membership, which is useful for targeting specific teams or on-call rotations. Use this method when you want to notify only the people currently on call or a specific group of responders.

To configure Slack notifications in your escalation chains:

1. In Grafana Cloud, navigate to Alerts & IRM > IRM > Escalation chains.
2. Select an existing escalation chain or create a new one by clicking + Add escalation chain.
3. Use the dropdown under Add escalation step to choose either Notify all Slack channel members or Notify Slack user group.
4. Select the appropriate Slack channel or user group from the dropdown.
5. Configure any additional escalation steps and save your changes.

Route alerts to different Slack channels

To route different alerts to different Slack channels, create unique escalation chains with the necessary Slack notification steps and the associated Slack channels or user groups.

Then configure Routes to direct alerts to the appropriate escalation chain. Routes can be configured to determine which escalation chain to use based on alert labels and conditions.

For example, if you want to send database alerts to a #database-team channel and API alerts to an #api-team channel:

1. Create an escalation chain called “Database Escalation” with a step that notifies all Slack channel members in #database-team.
2. Create an escalation chain called “API Escalation” with a step that notifies all Slack channel members in #api-team.
3. Create a Route that matches alerts with service="database" and directs them to the “Database Escalation” chain.
4. Create a Route that matches alerts with service="api" and directs them to the “API Escalation” chain.

Configure user notification rules with Slack mentions

User notification rules let you control when and how you’re notified of alerts from IRM. These notifications are separate from escalation chain notifications and are based on individual users settings and personal preferences.

You can configure your notifications rules to receive personal Slack mentions for alerts that have been escalated to you.

Before configuring Slack notifications, ensure you have connected your Slack account as described in Connect your Slack profile to Grafana IRM.

To configure Slack mentions for your notification rules:

1. Go to your Grafana Cloud Profile.
2. Select the IRM tab.
3. Click Notification rules.
4. Click + Add notification rule.
5. Choose Slack mentions from the notification channel dropdown.
6. Configure the remaining notification rule settings as needed, such as notification importance and conditions.

Your personal notification rules work independently from escalation chain notifications.

For example, you might be receive a Slack notification for an alert that was routed through an escalation chain that notifies a Slack channel, even if you don’t have Slack notifications as part of your notifications rules.

If you don’t see Slack as an available notification channel in your user profile, it may be disabled by your organization administrator. For more information about notification channel visibility and management, refer to the Notification channel documentation.

Configure on-call shift notifications

The IRM integration for Slack supports automated notifications for on-call shifts, helping teams stay informed of their on-call duties. You can also configure IRM to automatically update Slack user group membership to reflect current on-call assignments, ensuring that mentions of the user group always reach the people currently on call.

Admins can set up these notifications either in Grafana IRM or directly from Slack.

Configure on-call shift notifications in Grafana IRM

To configure on-call shift notifications for a schedule:

1. In Grafana Cloud, navigate to Alerts & IRM > IRM > Schedules.
2. Find the schedule you want to configure, and click the gear icon to open Schedule settings.
3. Expand the Notifications settings section.
4. Configure the notification behavior for future shifts according to your team’s preferences.
5. Optionally, select a Slack user group to automatically update with current on-call assignments. When configured, IRM updates the user group membership whenever on-call rotations change.
6. Click Update Schedule to save your changes.

Configure on-call shift notifications directly in Slack

When you receive an on-call shift notification in Slack, you can configure notification preferences directly from the notification message:

1. When an on-call shift notification is sent to a person or channel, click the gear icon to open Notifications preferences.
2. Configure the notification behavior for future shifts according to your team’s preferences.
3. Click Close.

These preferences are saved and apply to future shift notifications for the same schedule.

Configure incident Slack channels

You can configure IRM to automatically create dedicated Slack channels for incidents and post incident updates to those channels. This section outlines incident-specific Slack channel settings you can configure to suit your organization’s incident response process.

Configure incident channel creation

Define when dedicated incident Slack channels are created in to fit your organization’s workflows.

To configure automatic incident channel creation:

1. In Grafana Cloud, navigate to IRM > Integrations > Apps and select Slack.
2. Specify the following settings:
   • Create Slack channel: Determine when to automatically create a Slack channel for an incident.
   • Post updates to channel: Choose whether to automatically post incident updates to the created Slack channel.
   • Invite users: Specify if users with an active role should be automatically invited to the incident channel.

For private incidents, specify a default private Slack channel to post updates about any ongoing private incidents.

Configure automatic channel archive

To maintain good Slack workspace hygiene, you can automatically archive resolved incident channels after a configurable number of days.

Consider a timeframe of 30-60 days to allow time for post-incident reviews, follow-ups, and stakeholder visibility.

To configure automatic channel archive:

1. In Grafana Cloud, navigate to IRM > Integrations > Apps and select Slack.
2. Set the desired number of days after incident resolution for channels to automatically archive.
3. Click Save settings.

Configure channel name template

You can customize incident Slack channel names by configuring the channel name template. This allows you to standardize channel naming conventions across your organization.

By default, the template is: {{ prefix }}-{{ date }}-{{ title }}

This creates channel names like incident-2024-01-12-database-outage.

Available template variables:

• {{ prefix }} - The incident prefix, configurable in incident settings
• {{ title }} - The incident title, automatically slugified for Slack compatibility
• {{ date }} - The incident creation date in YYYY-MM-DD format
• {{ id }} - The unique incident ID

To configure the channel name template:

1. In Grafana Cloud, navigate to IRM > Integrations > Apps and select Slack.
2. In the Channel name template field, enter your desired template using the available variables.
3. Click Save settings.

Updating the channel name template only affects incidents created after the change. Existing incident channels aren’t renamed retroactively.

Example templates:

• {{ prefix }}-{{ date }}-{{ title }} produces incident-2025-01-15-database-outage
• {{ title }}-{{ id }} produces database-outage-inc-123456
• ops-{{ title }} produces ops-database-outage

Customize Slack channel prefixes

You can define custom channel prefixes to organize and identify incident channels more effectively.

To customize Slack channel prefixes:

1. In Grafana Cloud, navigate to Alerts & IRM > IRM > Settings.
2. Select the Incidents tab and find the Prefixes section.
3. Click + Add Prefix to create a new prefix.
4. Enter a Name and Description for the prefix.
5. Edit or delete existing prefixes as needed.
6. When declaring a new incident, select the desired prefix from the dropdown menu.

For more information, refer to the Customize incident channel prefixes documentation.

Configure incident announcements

Incident announcements keep your team aware and informed during critical events by posting notifications to designated Slack channels when incidents are declared.

Incident announcements are separate from the incident-specific Slack channels that IRM creates automatically. Announcements are posted to a designated notification channel to inform a broader audience, while incident channels serve as dedicated workspaces for incident response.

You can configure which incidents trigger announcements and which channels receive them based on incident type, severity, and custom filters.

For more information and configuration details, refer to the Incident announcement documentation.

Use Slack reactions

You can use emoji reactions to add context or updates to an incident without leaving Slack. Reactions provide a quick way to capture important information during incident response.

The following reactions are supported:

Additional details and limitations:

• Removing the 🤖 (:robot:) reaction deletes any attached files from Grafana Cloud storage.
• Deleting an item from the timeline within the IRM web app also removes the associated file from Grafana Cloud storage.
• Files in Grafana IRM are limited to 100MB. Files too large to persist are still accessible via a link to the source file in Slack.
• You can’t add new status updates to incidents that have been resolved.

Available commands for Slack

The /grafana commands for Slack allow users and teams to respond to alert groups and collaborate on incidents directly from Slack.

Global commands can be used in any Slack channel, but some commands are only available within incident-specific channels.

To use /grafana commands, you must be an IRM user with a linked Slack account and necessary permissions. The action you can take depends on your permissions in Grafana IRM.

Incident-related commands

Incidents management

Manage incidents from any Slack channel using the following commands:

In-Channel Incident Management

Manage the current incident from within the incident-specific Slack channel with the following commands:

Tasks

Manage tasks for an incident from within the incident-specific Slack channel:

Notes

Manage notes for an incident from within the incident-specific Slack channel:

Work with multiple stacks

If you connect multiple Grafana Cloud stacks to one Slack workspace, you need to manage which stack receives your slash commands and where notifications are sent. Slash commands and notifications use your default stack unless you explicitly specify a different one.

When you have multiple stacks connected to your Slack workspace:

• Each user must set a default stack.
• Slash commands use your default stack unless you explicitly specify a different one.
• Notifications from each stack are independent and post to the channels configured in that stack’s settings.
• You can switch between stacks at any time.

To view all available stacks and set your default:

1. In any Slack channel, run /grafana stacks or /grafana set-stack.
2. Select your desired stack from the selection menu that appears.
3. Confirm that your default stack is set by running any /grafana command.

You can also set your default stack by typing the stack handle directly using /grafana set-stack <stack>.

If you’re not seeing expected behavior from slash commands, verify your default stack is correct by running /grafana stacks.

Troubleshoot IRM Slack issues

For general Slack integration guidance, installation problems, or permission issues, refer to the Troubleshooting section in the Grafana Cloud app for Slack documentation.

If you encounter issues with Slack notifications or incident collaboration in IRM, try the following troubleshooting steps:

Not receiving Slack notifications from escalation chains

If you’re not receiving Slack notifications for alerts:

• Verify that the escalation chain includes a Slack notification step. Check the escalation chain configuration in Alerts & IRM > IRM > Escalation chains.
• Ensure the Grafana Cloud app has been added to the Slack channel configured in the escalation chain.
• For user group notifications, verify that the user group exists in your Slack workspace and contains the expected members.
• Check that your Route is directing alerts to the correct escalation chain.

Not receiving personal Slack mentions

If you’re not receiving personal Slack mentions for alerts based on your notification rules:

• Verify your Slack account is linked to your Grafana Cloud account. Refer to Connect your Slack profile to Grafana IRM.
• Check your notification rules in your Grafana Cloud profile under the IRM tab to ensure Slack mentions are configured.
• Verify that Slack is enabled as a notification channel by your organization administrator. Refer to the Notification channel documentation.

Slash commands not working

If /grafana commands aren’t working in Slack:

• Verify your Slack account is linked to your Grafana Cloud account. Refer to Connect your Slack profile to Grafana IRM.
• Check that you have the necessary permissions in Grafana IRM to perform the action you’re attempting.
• Ensure you’re using the commands in the correct context. Some commands, for example incident management commands, only work within incident-specific Slack channels.
• If you have multiple stacks connected, verify your default stack is correct using /grafana stacks. You may need to set your default stack using /grafana set-stack <stack>.

Can’t create or access private incident channels

If you’re having issues with private incident channels:

• Verify that you’ve created a private Slack channel manually and invited the Grafana Cloud app to that channel. IRM can’t automatically create private Slack channels due to Slack’s permission model.
• Ensure the private channel is configured as the default private incident channel in IRM > Integrations > Apps > Slack.
• Check that users who need access to private incidents have been added to the private Slack channel.

Rate limiting issues

If you see rate limiting messages in Slack:

• Slack enforces rate limits per channel and per organization. During high alert activity, IRM may temporarily hit these limits.
• When rate-limited, IRM blocks posting or updating alert groups for 5 minutes, then automatically resumes.
• Consider reducing alert volume or consolidating notifications to avoid frequent rate limiting.

Rate limits also affect personal Slack notifications:

• If an alert group can’t be posted due to rate limiting, related personal notifications in Slack are skipped.
• In some cases, even if the alert group is successfully posted, Slack may still block the personal notification due to rate limits.

For more information about rate limiting, refer to the Rate limiting section in the Grafana Cloud app for Slack documentation.