Configure incident announcements

Incident announcements help keep your team aware and informed during critical events by automatically posting incident information to designated Slack channels. Configure these announcements to ensure that stakeholders are kept up-to-date in Slack during an incident without needing to monitor multiple channels or dashboards.

When an incident is declared, announcement messages are automatically posted to your configured Slack channels. These messages include key incident details such as severity, status, assigned roles, and labels. Announcement messages are kept up-to-date as the incident changes, ensuring your team always has the latest information.

Before you begin

To configure incident announcements, you need:

A Grafana Cloud account with IRM permissions
The Grafana IRM integration for Slack installed and configured
Access to the Slack workspace where you want to send announcements

For information about installing the Slack integration, refer to the Grafana IRM integration for Slack documentation.

Configure incident announcements

You can configure multiple announcement channels, each with its own filtering criteria, to route different types of incidents to the appropriate teams or stakeholders.

To configure incident announcements:

In Grafana Cloud, navigate to IRM > Integrations > Apps and select Slack.
Select the Slack channel where you want to send notifications. You can either choose from the dropdown menu or manually add your Channel ID. The Channel ID can be found in the About tab of the Slack channel.
Define the incident-specific fields:
- Incident type: Choose whether the incident is internal or private.
- Include incidents: Specify which types of incidents to announce—options include all incidents, drills, or non-drills.
Apply filters to your incident announcements to tailor the notifications to specific channels:
- Incident filter: Use incident query syntax to filter by various properties such as labels, severity, status, or custom fields. By default, all conditions are applied using AND logic. To use OR logic, you must use parentheses. For example: or(label:"squad:team-a" label:"team-b") status:active filters for (team-a OR team-b) AND status:active.

For complete information about incident query syntax, refer to the Incident query syntax documentation.

Route incident announcements by filter

Configure announcements with different filters to route incident announcements to the appropriate teams or stakeholders. This approach helps ensure that the right people are notified about incidents relevant to their work.

For example:

#incidents-datasources: Filter label:"squad:datasources" to notify the datasources team channel about incidents affecting their services.
#incidents-critical: Filter severity:critical to post in a designated channel for informing stakeholders about critical incidents.
#incidents-security: Filter label:"security" to notify the necessary stakeholders about security-related incidents.

By configuring different announcement channels for specific types of incidents based on labels, severity, or other criteria, team members and stakeholders can join the announcement channels appropriate to them. Establish these channels as central incident communication hubs across your teams and stakeholders.

All channel members can manage their Slack channel notification preferences. Channel members can subscribe to the announcement thread using Slack’s Get notified of new replies setting to receive status update notifications for incidents they want to follow more closely.

This approach allows team members to:

Stay informed about incidents relevant to their work.
Receive notifications only for incidents that match their interests.
Follow incident updates through threaded replies.

Status update notifications

When status updates are posted to an incident, they’re also posted as threaded replies to the announcement message in Slack. This keeps all incident updates organized in one place and makes it easier for stakeholders to follow the incident’s progress.

Stakeholders can subscribe to the announcement thread using Slack’s Get notified of new replies setting to receive notifications about new status updates.

To enable status updates in announcement threads, ensure you have the latest version of the Grafana IRM integration for Slack installed. If status updates aren’t appearing in threads, you may need to reinstall the integration. For more information, refer to Reinstall the Grafana IRM integration for Slack.

Best practices

Consider these recommendations when configuring incident announcements:

Use specific filters: Create targeted announcement channels with specific filters rather than announcing all incidents to a single channel. This reduces noise and ensures team members only see incidents relevant to them.
Organize by team or service: Create separate announcement channels for different teams or services using label-based filters. This helps teams focus on incidents that affect their area of responsibility.
Use severity-based routing: Create announcement channels filtered by severity level to ensure critical incidents reach the right stakeholders quickly.
Leverage threaded updates: Encourage team members and stakeholders to subscribe to announcement threads to receive status update notifications automatically.
Separate drill and production incidents: Consider creating separate announcement channels for drills to avoid adding noise to real incidents channels.