Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Platformbreadcrumb arrow Workflowsbreadcrumb arrow Referencebreadcrumb arrow Blueprintsbreadcrumb arrow AI alert triage
Public preview Grafana Cloud
Last reviewed: November 9, 2026

AI alert triage

Note

Grafana Workflows is currently in public preview. Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available.

Grafana Assistant investigates every new alert group, posts a triage summary to Slack, and declares an incident when it decides the alert needs escalating.

This Grafana Workflows blueprint is in the Alerts category of the Blueprints library.

Before you begin

To use this blueprint, you need:

  • Grafana IRM set up on your stack.
  • Slack connected to Grafana IRM. For details, refer to Configure Slack for Grafana IRM.
  • Grafana Assistant in use on your stack.

Steps

Steps run in order from first to last. A step inside a switch branch runs only when Workflows chooses that branch.

StepIDTypeDescription
Investigate the alerttriageassistant.promptAsks Grafana Assistant to investigate and recommend whether to escalate. Edit the escalation criteria to match your team’s bar for an incident.
Escalate or reportrouteswitchDeclares an incident when the assistant recommends escalation, otherwise only posts the triage. No changes needed.
Declare an incident (branch 1 of route)declareincident.createCreates an incident for the alert. Change the severity to match how you classify escalated alerts.
Add the triage to the incident (branch 1 of route)note-triageincident.add-noteRecords the assistant’s findings on the new incident’s timeline. No changes needed.
Announce the escalation (branch 1 of route)post-escalatedslack.message.postPosts the escalation to your incident channel. Set channelID.
Post the triage summary (default branch of route)post-triageslack.message.postPosts the triage for alerts that do not need an incident. Set channelID to your team’s alerts channel.

For the inputs and outputs of each step type, refer to Step and trigger types reference.

Before you turn on the workflow, customize any steps that need input from you, such as a channel ID or a schedule name. Each step’s description says what to set.

Workflow definition

Save this definition as a YAML file to create the workflow with gcx, the Workflows API, or Terraform, or give it to a coding agent as an example. Replace FOLDER_UID with the UID of the folder for the workflow. Blueprints start disabled, so finish customizing the steps before you set enabled to true. For details, refer to Manage workflows as code.

YAML 
apiVersion: workflows.ext.grafana.app/v1alpha1
kind: Definition
metadata:
  name: ai-alert-triage
  annotations:
    grafana.app/folder: <FOLDER_UID>
spec:
  name: AI alert triage
  enabled: false
  runOnceFor: 'triage:${inputs.data.alertGroupID}'
  startWhen:
    matchingRules:
      - eventNameRegex: 'grafana_irm_app\.alertgroup\.updated'
        filter: "inputs.data.eventKind == 'created'"
    schedules: []
  steps:
    - id: triage
      type: assistant.prompt
      name: Investigate the alert
      description: Asks Grafana Assistant to investigate and recommend whether to escalate. Edit the escalation criteria to match your team's bar for an incident.
      inputs:
        prompt: |
          A new alert group fired in Grafana IRM. Investigate it and triage it.

          Title: ${resources.alertgroup.event.title}
          Alert group: ${resources.alertgroup.event.permalinks.web}
          Labels: ${json_stringify(resources.alertgroup.event.labels)}
          Alerts in group: ${resources.alertgroup.event.alerts_count}

          Look at the related metrics, logs, and traces from the last hour, recent deploys if visible, and similar alert groups from the past week.
          Set escalate to true only when users are likely affected right now, or the problem is getting worse quickly.
          Keep summary under 400 characters, written for an engineer who has 30 seconds.
        outputSchema: |
          {
            "type": "object",
            "required": ["summary", "likelyCause", "impact", "nextSteps", "escalate"],
            "properties": {
              "summary": { "type": "string" },
              "likelyCause": { "type": "string" },
              "impact": { "type": "string", "enum": ["none", "low", "medium", "high"] },
              "nextSteps": { "type": "array", "items": { "type": "string" } },
              "escalate": { "type": "boolean" }
            }
          }
        maxRetries: '3'
    - id: route
      type: switch
      name: Escalate or report
      description: Declares an incident when the assistant recommends escalation, otherwise only posts the triage. No changes needed.
      branches:
        - condition: 'steps.triage.outputs.result.escalate == true'
          steps:
            - id: declare
              type: incident.create
              name: Declare an incident
              description: Creates an incident for the alert. Change the severity to match how you classify escalated alerts.
              inputs:
                title: '${resources.alertgroup.event.title}'
                severity: 'major'
            - id: note-triage
              type: incident.add-note
              name: Add the triage to the incident
              description: Records the assistant's findings on the new incident's timeline. No changes needed.
              inputs:
                incidentID: '${steps.declare.outputs.incidentID}'
                message: |
                  **AI triage of [${resources.alertgroup.event.title}](${resources.alertgroup.event.permalinks.web})**

                  ${steps.triage.outputs.result.summary}

                  **Likely cause:** ${steps.triage.outputs.result.likelyCause}
                  **Impact:** ${steps.triage.outputs.result.impact}
                  **Next steps:** ${json_stringify(steps.triage.outputs.result.nextSteps)}
            - id: post-escalated
              type: slack.message.post
              name: Announce the escalation
              description: Posts the escalation to your incident channel. Set channelID.
              inputs:
                channelID: ''
                messageText: |
                  :rotating_light: *Escalated to incident ${steps.declare.outputs.incidentID}:* <${resources.alertgroup.event.permalinks.web}|${resources.alertgroup.event.title}>
                  ${steps.triage.outputs.result.summary}
        - steps:
            - id: post-triage
              type: slack.message.post
              name: Post the triage summary
              description: Posts the triage for alerts that do not need an incident. Set channelID to your team's alerts channel.
              inputs:
                channelID: ''
                messageText: |
                  :mag: *Triage:* <${resources.alertgroup.event.permalinks.web}|${resources.alertgroup.event.title}> (impact: ${steps.triage.outputs.result.impact})
                  ${steps.triage.outputs.result.summary}
                  *Likely cause:* ${steps.triage.outputs.result.likelyCause}

How it works

This section explains when the workflow runs and how its steps work together.

When it runs

This blueprint starts on the grafana_irm_app.alertgroup.updated event.

Its trigger filter starts a run only when this CEL expression is true:

text 
inputs.data.eventKind == 'created'

It runs at most once for each run-once key. This blueprint’s key is triage:${inputs.data.alertGroupID}, so a later event with the same key doesn’t start another run. For details, refer to Deduplicate workflow runs.

How the steps fit together

These notes cover how the steps connect and which Workflows behaviors the blueprint relies on.

  • Triggers on grafana_irm_app.alertgroup.updated with eventKind created.
  • runOnceFor keys on the alert group ID.
  • The assistant.prompt step sends the alert title, labels, and link to Grafana Assistant, which can query metrics, logs, traces, and IRM, and asks for JSON that matches outputSchema, including a boolean escalate.
  • The switch reads steps.triage.outputs.result.escalate.
  • The escalate branch declares an incident, notes the triage on its timeline, and posts to Slack.
  • The default branch posts the triage summary only.
  • Steps inside a branch can read prior top-level steps, such as triage, and earlier steps in the same branch, such as declare.
  • Narrow the trigger filter, for example by team_id or labels, before you enable it on a busy stack.
  • Customize the escalation criteria in the prompt, the incident severity, and the Slack channels.

Next steps