--- title: "Major incident kickoff | Grafana Cloud documentation" description: "Spin up a major incident in seconds. Creates the incident Slack channel, assigns the on-call incident commander, posts a help message, and bookmarks the runbook." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Major incident kickoff > Note > > Grafana Workflows is currently in [public preview](https://grafana.com/docs/release-life-cycle/). Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available. Spin up a major incident in seconds. Creates the incident Slack channel, assigns the on-call incident commander, posts a help message, and bookmarks the runbook. This Grafana Workflows blueprint is in the **Incidents** category of the Blueprints library. ## Before you begin To use this blueprint, you need: - Grafana IRM set up on your stack. - Slack connected to Grafana IRM. For details, refer to [Configure Slack for Grafana IRM](https://grafana.com/docs/grafana-cloud/alerting-and-irm/irm/integrations/chat-and-collaboration/slack/). ## Steps Steps run in order from first to last. A step inside a switch branch runs only when Workflows chooses that branch. Expand table | Step | ID | Type | Description | |----------------------------------------------------------------------------|---------------------|---------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Create incident channel | `create-channel` | `incident.slack-channel.create` | Creates a Slack channel linked to the incident and invites everyone who already holds an incident role. IRM posts incident activity to the channel. Change `channelName` to match your naming convention. Slack naming rules are applied automatically. | | Find the on-call commander | `get-on-call` | `irm.schedule-current-oncall` | Looks up who is currently on call in your incident commander rotation. Set `scheduleIdentifier` to the exact name of the IRM schedule, or switch `identifierType` to id and paste the schedule ID. | | Assign commander if someone is on call | `assign-commander` | `switch` | Assigns the first on-call user as incident commander. When nobody is on call, the default branch leaves a note on the timeline so the team assigns one manually. | | Make on-call user the commander (branch 1 of `assign-commander`) | `add-commander` | `incident.add-participant` | Gives the on-call user the commander role on the incident. No changes needed. | | Note that no commander was assigned (default branch of `assign-commander`) | `note-no-commander` | `incident.add-note` | Records that nobody was on call so the team knows to pick a commander. No changes needed. | | Post help message | `post-help` | `slack.message.post` | Posts a help message to the new channel that explains roles and how to work the incident. Edit the text to match your major incident process. | | Bookmark the runbook | `bookmark-runbook` | `slack.channel.add-bookmark` | Adds your major incident runbook to the channel bookmarks. Set `bookmarkUrl` to your runbook link. | | Bookmark the incident page | `bookmark-incident` | `slack.channel.add-bookmark` | Adds the Grafana IRM incident page to the channel bookmarks. No changes needed. | | Record the kickoff on the timeline | `note-kickoff` | `incident.add-note` | Leaves a timeline note that links the new channel. No changes needed. | For the inputs and outputs of each step type, refer to [Step and trigger types reference](https://grafana.com/docs/grafana-cloud/platform/workflows/reference/step-trigger-types). Before you turn on the workflow, customize any steps that need input from you, such as a channel ID or a schedule name. Each step’s description says what to set. ## Workflow definition Save this definition as a YAML file to create the workflow with gcx, the Workflows API, or Terraform, or give it to a coding agent as an example. Replace *FOLDER\_UID* with the UID of the folder for the workflow. Blueprints start disabled, so finish customizing the steps before you set `enabled` to `true`. For details, refer to [Manage workflows as code](https://grafana.com/docs/grafana-cloud/platform/workflows/as-code). YAML ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy ```yaml apiVersion: workflows.ext.grafana.app/v1alpha1 kind: Definition metadata: name: major-incident-kickoff annotations: grafana.app/folder: spec: name: Major incident kickoff enabled: false runOnceFor: 'major-incident:${inputs.data.incidentID}' startWhen: matchingRules: - eventNameRegex: 'grafana_irm_app\.incident\.updated' filter: "resources.incident.event.status == 'active' && resources.incident.event.severity.matches('(?i)^critical$')" schedules: [] steps: - id: create-channel type: incident.slack-channel.create name: Create incident channel description: >- Creates a Slack channel linked to the incident and invites everyone who already holds an incident role. IRM posts incident activity to the channel. Change channelName to match your naming convention. Slack naming rules are applied automatically. inputs: incidentID: '${inputs.data.incidentID}' channelName: 'incident-${inputs.data.incidentID}' postUpdates: 'true' inviteUsers: 'true' - id: get-on-call type: irm.schedule-current-oncall name: Find the on-call commander description: >- Looks up who is currently on call in your incident commander rotation. Set scheduleIdentifier to the exact name of the IRM schedule, or switch identifierType to id and paste the schedule ID. inputs: scheduleIdentifier: '' identifierType: 'name' - id: assign-commander type: switch name: Assign commander if someone is on call description: >- Assigns the first on-call user as incident commander. When nobody is on call, the default branch leaves a note on the timeline so the team assigns one manually. branches: - condition: "steps['get-on-call'].outputs.count > 0" steps: - id: add-commander type: incident.add-participant name: Make on-call user the commander description: Gives the on-call user the commander role on the incident. No changes needed. inputs: incidentID: '${inputs.data.incidentID}' userID: "${steps['get-on-call'].outputs.users[0].email}" role: 'commander' - steps: - id: note-no-commander type: incident.add-note name: Note that no commander was assigned description: Records that nobody was on call so the team knows to pick a commander. No changes needed. inputs: incidentID: '${inputs.data.incidentID}' message: 'Nobody was on call in the incident commander schedule, so no commander was assigned automatically. Assign one from the incident page.' - id: post-help type: slack.message.post name: Post help message description: >- Posts a help message to the new channel that explains roles and how to work the incident. Edit the text to match your major incident process. inputs: channelID: "${steps['create-channel'].outputs.channelID}" messageText: | :rotating_light: *Major incident declared: ${resources.incident.event.title}* *Commander:* ${steps['get-on-call'].outputs.count > 0 ? steps['get-on-call'].outputs.users[0].username : 'not assigned yet, pick one now'} *Severity:* ${resources.incident.event.severity} *Incident page:* ${resources.incident.event.overviewURL} *How we work this incident* • The commander coordinates. Ask before you take any risky action. • Keep the discussion in this channel. Use threads for side investigations. • Post key updates on the incident page so stakeholders stay informed. • Follow the runbook bookmarked at the top of this channel. - id: bookmark-runbook type: slack.channel.add-bookmark name: Bookmark the runbook description: Adds your major incident runbook to the channel bookmarks. Set bookmarkUrl to your runbook link. inputs: channelID: "${steps['create-channel'].outputs.channelID}" bookmarkTitle: 'Major incident runbook' bookmarkUrl: '' bookmarkEmoji: ':book:' - id: bookmark-incident type: slack.channel.add-bookmark name: Bookmark the incident page description: Adds the Grafana IRM incident page to the channel bookmarks. No changes needed. inputs: channelID: "${steps['create-channel'].outputs.channelID}" bookmarkTitle: 'Incident page' bookmarkUrl: '${resources.incident.event.overviewURL}' bookmarkEmoji: ':fire:' - id: note-kickoff type: incident.add-note name: Record the kickoff on the timeline description: Leaves a timeline note that links the new channel. No changes needed. inputs: incidentID: '${inputs.data.incidentID}' message: "Major incident kickoff ran. Channel: #${steps['create-channel'].outputs.channelName}" ``` ## How it works This section explains when the workflow runs and how its steps work together. ### When it runs This blueprint starts on the `grafana_irm_app.incident.updated` event. Its trigger filter starts a run only when this CEL expression is true: text ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy ```text resources.incident.event.status == 'active' && resources.incident.event.severity.matches('(?i)^critical$') ``` It runs at most once for each run-once key. This blueprint’s key is `major-incident:${inputs.data.incidentID}`, so a later event with the same key doesn’t start another run. For details, refer to [Deduplicate workflow runs](https://grafana.com/docs/grafana-cloud/platform/workflows/build/triggers/#deduplicate-workflow-runs). ### How the steps fit together These notes cover how the steps connect and which Workflows behaviors the blueprint relies on. - Runs once per incident when an active incident is Critical, whether it was declared Critical or raised to Critical later. - Incident events carry only IDs, so the trigger filter reads the auto-loaded incident at `resources.incident.event` (trigger filters run after resources load). - `runOnceFor` keys on the incident ID. - Because the severity check lives in the trigger filter rather than in a filter step, events that do not match release the run-once key, so a Pending incident that is later raised to Critical still fires exactly once. - Steps: create a Slack channel linked to the incident (`incident.slack-channel.create`, so IRM posts updates there), look up who is on call in the incident commander schedule, assign that person the commander role when someone is on call (switch with a default branch that leaves a timeline note when nobody is), post a help message, bookmark the runbook and the incident page, and record a timeline note. - Steps after the switch reference get-on-call, not the branch steps, because branch step outputs are not visible outside their branch. - Customize the schedule name, the runbook URL, the severity regex in the trigger filter, and the help message. - If your IRM Slack integration already creates a channel for every incident, delete create-channel and use `resources.incident.event.slackChannelID` instead. ## Next steps - [Blueprints](https://grafana.com/docs/grafana-cloud/platform/workflows/reference/blueprints) - [Start from a blueprint](https://grafana.com/docs/grafana-cloud/platform/workflows/build/use-blueprints) - [Manage workflows as code](https://grafana.com/docs/grafana-cloud/platform/workflows/as-code) - [Step and trigger types reference](https://grafana.com/docs/grafana-cloud/platform/workflows/reference/step-trigger-types)