Documentation Grafana Cloud Alerts and IRM IRM IRM Slack integration

Slack integration for Grafana IRM

The Grafana IRM Slack integration incorporates your incident response workflow directly into your Slack workspace, helping your team focus on alert resolution with less friction.

Key features and benefits

Integrating your Slack workspace allows users and teams to be notified of alerts directly in Slack with automated alert escalation steps and user notification preferences. Users can take a number of alert actions directly from Slack, including acknowledging and resolving alert groups, adding resolution notes, and more.

Refer to the following sections to learn more:

• Configure Escalation Chains with Slack notifications
• Configure user notifications policies with Slack mentions
• Configure on-call notifications in Slack

When it comes to incidents, users can declare, collaborate on, and manage incident-worthy events without leaving Slack. They can also automatically create incident-specific channels, track the timeline of events, interact with incidents via Slash commands, and more.

Refer to the following sections to learn more:

• Create incident Slack channels
• Configure incident announcements
• Interact with incidents from Slack via Slash commands

Before you begin

To install the Grafana IRM Slack app, you must:

• Have an Admin role in Grafana IRM
• Be a Slack workspace Admin or Owner
• Allow Grafana IRM to access your Slack workspace

After the IRM Slack app has been added to your Slack workspace, Admins and Editors can configure escalation chains and other notifications to route to designated Slack channels.

In order to use the Slack slash command /grafana, you must be an IRM user with a linked Slack account and necessary permissions.

To learn more, refer to Connect your Slack user to Grafana IRM.

Install the Grafana IRM Slack integration

Currently, the Grafana IRM Slack integration can be installed from Grafana OnCall or Grafana Incident. Installing the integration in either location automatically installs it for the other.

Follow these steps to install the Slack integration from Grafana OnCall:

1. Navigate to the Settings tab in Grafana OnCall.
2. From the ChatOps tab, select Slack in the side menu.
3. Click Install integration.
4. After being redirected to the Slack connection page, verify the Slack workspace is correct or Add another workspace.
5. If needed, provide your Slack workspace URL and sign with your Slack credentials.
6. Follow the Slack prompts to review permissions and specify a default channel.
7. Click Allow to give Grafana IRM permission to access your Slack workspace.

Follow these steps to install the Slack integration from Grafana Incident:

1. Navigate to the Integrations tab in Grafana Incident.
2. Click the Slack IRM tile and then click Install Integration.
3. After being redirected to the Slack connection page, verify the Slack workspace is correct or Add another workspace.
4. If needed, provide your Slack workspace URL and sign with your Slack credentials.
5. Follow the Slack prompts to review permissions and specify a default channel.
6. Click Allow to give Grafana IRM permission to access your Slack workspace.

For more information about the required permissions, refer to the Permissions section.

Connect your Slack user to Grafana IRM

For users to gain full access to Grafana IRM functionality in Slack, follow these steps to map your Grafana IRM user account to your Slack user account:

1. In Grafana OnCall, navigate to the Users tab and click View my profile.
2. Under the User Info tab, find the Slack username section and click Connect.
3. Follow the prompts to verify and link your Slack account.

After linking your Slack account, you can start using Grafana IRM in Slack in accordance with the role and permissions assigned to you in Grafana. Meaning the action you can take in Slack is dependent on your permissions in Grafana IRM. For more information on roles and permissions, refer to the Grafana Cloud user roles and permissions documentation.

Configure channels and notifications

Grafana IRM provides flexible configuration options to tailor Slack notifications to your team’s needs. This section outlines how to set up escalation chains, user notifications, on-call notifications, and incident channel settings within Slack.

Configure Escalation Chains with Slack notifications

After setting up your Slack integration, you can configure escalation chains to send notifications via Slack for alerts in Grafana OnCall.

There are two Slack notification methods that can be integrated into escalation chains:

• Notify all members of a Slack channel
• Notify a specific Slack user group

To configure these in your escalation chains:

1. In Grafana OnCall, navigate to the Escalation Chains tab.
2. Select an existing escalation chain or create a new one by clicking + New escalation chain.
3. Use the dropdown under Add escalation step to choose and configure your Slack notification preferences.

Configure user notifications policies with Slack mentions

You can receive alert notifications directly via Slack mentions, ensuring that critical alerts reach you immediately:

1. From the Users tab in Grafana OnCall and click Edit next to the user you want to configure.
2. In the User Info tab, click + Add Notification step.
3. Choose Notify by in the first dropdown, then select Slack mentions in the second dropdown to receive notifications through Slack mentions.

Configure on-call notifications in Slack

The IRM Slack integration also supports automated notifications for on-call shifts, helping teams stay informed of their duties. Admins can set up these notifications in the Notification Preferences section:

1. When an on-call shift notification is sent to a person or channel, click the gear icon to open Notifications preferences.
2. Configure the notification behavior for future shifts according to your team’s preferences.

Configure Incident Slack channels

Configure the following Incident-specific Slack channel settings to suit your organizations process and preferences.

Incident channel creation

Customize when incident Slack channels are created to align with your organization’s workflows.

To configure incident channel creation:

1. Navigate to the Slack IRM Integration page in Grafana Incident.
2. Specify the following settings:
   • Create Slack channel: Determine when to automatically create a Slack channel for an incident.
   • Post updates to channel: Choose whether to automatically post incident updates to the created Slack channel.
   • Invite users: Specify if users with an active role should be automatically invited to the incident channel.

Automatic channel archive

To maintain good Slack workspace hygiene, you can automatically archive resolved incident channels after a configurable numbers of days.

Consider a timeframe of 30-60 days to allow time for PIR, follow-ups, and stakeholder visibility.

To configure automatic channel archive:

1. Navigate to the Slack IRM Integration page in Grafana Incident.
2. Set the desired number of days after incident resolution for channels to automatically archive.
3. Click Save settings.

Customize Slack channel prefixes

You can define custom channel prefixes to organize and identify incident channels more effectively.

To customize Slack channel prefixes:

1. Navigate to Settings in Grafana Incident.
2. Scroll to the Prefixes section.
3. Click + Add Prefix to create a new prefix.
4. Enter a Name and Description for the prefix.
5. Edit or delete existing prefixes as needed.
6. When declaring a new incident, select the desired prefix from the dropdown menu.

Configure Incident announcements

Incident announcements help keep your team aware and informed during critical events. Configure these announcements to ensure that stakeholders are kept up-to-date in Slack during an incident.

To configure Incident announcements:

1. Navigate to Incident in the left-side menu, then click Integrations.
2. Click Slack IRM to open the Slack IRM integration page.
3. Select the Slack channel where you want to send notifications. You can either choose from the dropdown menu or manually add your Channel ID. The Channel ID can be found in the About tab of the Slack channel.
4. Define the incident-specific fields:
   • Incident type: Choose whether the incident is internal or private.
   • Include incidents: Specify which types of incidents to announce—options include all incidents, drills, or non-drills.
5. Apply filters to your incident announcements to tailor the notifications to specific channels:
   • Incident filter: For example, you might filter by label, such as label: 'squad:datasources'.

Manage Slack attachments

When you use the 🤖 emoji reaction on a Slack message containing a file, the file is securely copied to Grafana Cloud storage. This ensures your incident timeline remains intact, even if the file is later deleted from Slack.

• File retention: Control attachment retention in your incident timeline. Removing the 🤖 reaction deletes the attached files from Grafana Cloud storage.
• Incident web app: Deleting an item from the timeline within the Incident web app also removes the associated file from Grafana Cloud storage.
• File size limit: Files in Grafana Incident are limited to 100MB. Files too large to persist are still accessible via a link to the source file in Slack.

Available Slack commands

The /grafana Slack commands allow users and teams to respond to alert groups and collaborate on incidents directly from Slack.

Incident-related commands

Permissions scope and purpose

The Grafana IRM Slack app only requests permissions that are essential to its proper function and integration with Slack. Refer to the Slack documentation for more information on permission scopes.

By granting access to the app, you are authorizing Grafana IRM the following permissions in your Slack workspace:

Workspace and user access

Public channel access

Private channel access

Message and conversation access