Menu
Grafana Cloud

Outgoing webhooks for Grafana IRM

Outgoing webhooks in Grafana IRM enable flexible integration with external systems by automatically sending event data related to alerts and incidents to your preferred tools and APIs.

Note

Grafana IRM outgoing webhooks are now managed from a unified interface and processed through a shared backend service. If you have existing outgoing webhooks configured through the legacy Incident outgoing webhooks integration, those webhooks remain fully functional. However, we recommend migrating to the shared IRM webhook experience to take advantage of the latest features and ensure long-term support. For more information, refer to the Migrate from Incident webhooks section of this document.

Configure an outgoing webhook

To create a webhook from the IRM UI:

  1. In Grafana Cloud, go to Alerts & IRM > IRM > Integrations.
  2. Select the Outgoing Webhooks tab.
  3. Click + Create an Outgoing Webhook.
  4. Choose a preset:
    • Simple – Sends basic alert group data (used for escalation steps).
    • Advanced webhook for alert groups – Customizable with additional settings and templating.
    • Advanced webhook for incidents – Customizable with additional settings and templating.
    • Grafana Sift for alert groups – Forwards alert group data to Grafana Sift.
  5. Complete the webhook configuration form.
  6. Click Create to save the webhook.

For information about configuring outgoing webhooks via Terraform, refer to the Terraform documentation.

Webhook configuration fields

The following fields are available when configuring an outgoing webhook:

FieldDescriptionRequired
NameA descriptive display nameYes
EnabledWhether the webhook is activeYes (default: true)
Assign to teamTeam that owns or manages the webhookNo
Trigger typeEvent that causes the webhook to executeYes
HTTP methodGET, POST, PUT, PATCH, DELETE, or OPTIONSYes (default: POST)
IntegrationsRestrict execution to specific integrationsNo
LabelsLabels to include in the payload bodyNo
Webhook URLDestination for the requestYes
HeadersOptional request headers (templating supported)No
UsernameHTTP Basic Auth usernameNo
PasswordHTTP Basic Auth passwordNo
AuthorizationSupports basic auth or custom headersNo
Trigger templateUsed to conditionally execute the webhookNo
Forward whole payloadSends the full event object as received by IRM instead of a custom bodyNo

Note

Your destination endpoints must respond within a 4-second timeout window.

Supported triggers

The following event types are available to trigger a webhook to execute.

Alert group events

EventDescription
Manual or escalation stepTriggered manually or as part of an escalation chain
Alert group createdA new alert group is created
AcknowledgedThe alert group was acknowledged
ResolvedThe alert group was resolved
SilencedThe alert group was silenced
UnsilencedThe alert group was unsilenced
UnresolvedThe alert group was reopened
UnacknowledgedAcknowledgment was removed
Status changeAny state change event
Resolution note addedA resolution note was added
Personal notificationTriggered by a personal notification rule

Incident events

EventDescription
Incident declaredA new incident is declared
Incident changedUpdates to title, severity, status, role, or labels
Incident resolvedThe incident is resolved
Manual from incidentTrigged manually via dropdown in incident Activity->Action Tab

Note

Each matching event triggers a webhook once. Removing and re-adding a matching label won’t retrigger the webhook.

Incident subevents

TriggerSubevent
Incident declaredgrafana.incident.created
Incident changedgrafana.incident.updated.role
grafana.incident.updated.status
grafana.incident.updated.title
grafana.incident.updated.severity
grafana.incident.added.label
grafana.incident.removed.label
Incident resolvedgrafana.incident.closed
Manual from incidentgrafana.incident.manual

Use webhook templates

Use Jinja2 templates to include dynamic content from the event context. Templates have access to context data from the triggering event. Supported fields vary by trigger type.

For more information about templates in IRM, refer to the Templates documentation.

Common template fields

  • {{ event.type }} – Type of event
  • {{ event.time }} – Timestamp
  • {{ user.username }} – Username if triggered by a user
  • {{ alert_group.title }} – Alert group title
  • {{ alert_payload }} – Contents of the first alert in the group
  • {{ incident.title }} – Incident title
  • {{ incident.severity }} – Incident severity
  • {{ webhook.name }} – Webhook name

Example templates

Basic JSON payload:

JSON
{
  "name": "{{ alert_payload.labels.alertname }}",
  "message": "{{ alert_payload.annotations.description }}"
}

URL with query params:

https://example.com/tickets?assignee={{ user.email }}

Use tojson() to format objects:

JSON
{
  "labels": {{ alert_payload.labels | tojson() }}
}

Reference previous responses

You can access results from previously executed webhooks using the {{ responses }} variable. This enables advanced workflows, such as updating a ticket that was created by an earlier webhook.

Note

There is a retention policy of 90 days for previous responses.

Conditional execution

Use the Trigger Template field to control when a webhook executes. The webhook will run only if the template evaluates to an empty string, True, or 1.

Example to trigger only for high-severity alerts:

{% if alert_payload.labels.severity == "high" %}true{% endif %}

Example to trigger only for incidents with “Critical” severity and the “Customer” label set:

{% if incident.severity == "Critical" and incident.labels | selectattr('key', 'equalto', 'Customer') | list | length > 0 %}true{% endif %}

View webhook execution results

You can view webhook execution history in the Outgoing Webhooks tab. The list includes the latest events and associated data.

Click a webhook to view detailed execution information:

  • The triggered event and context
  • Request details: URL, headers, and body
  • Response code and response body
  • Template evaluation output

You can also find webhook responses associated with specific events:

  • For alert group webhooks, a link to the corresponding response appears in the alert group timeline, accessible from the alert group detail page.
  • For incident webhooks, execution results appear in the incident timeline under the All activity view.

Migrate from Incident outgoing webhooks

Legacy incident webhooks continue to work but will eventually be deprecated in future releases.

To migrate a webhook from the legacy Incident outgoing webhooks app, manually recreate the webhook in the shared IRM Outgoing Webhooks interface with the following settings:

Name: <your webhook name>
HTTP method: POST
URL: <your URL>
Trigger type: <your original trigger type>
Customize forwarded data [x]
Data template: {{ incident }}

Be aware that the IRM shared webhook requests do not include signature or Cloud Events headers.

After enabling the corresponding IRM webhook, test your new webhook before removing the legacy webhook. Once verified, delete the legacy configuration to avoid duplicate executions.

If your webhook previously used the Incident matches filters trigger, use the Incident changed trigger type. If you want to respond to other incident events, such as declarations or resolutions, create additional webhooks for those trigger types as needed.

Recreate your filter logic using Jinja templating in the Trigger template field. This provides equivalent behavior to the original filtered trigger in the legacy Incident integration.

For example:

-severity:critical is equivalent to {{ incident.severity != "critical" }}

or

and(label:customer severity:critical) can be expressed as {{ incident.severity == "Critical" and incident.labels | selectattr('key', 'equalto', 'Customer') | list | length > 0 }}

Examples and integrations

Outgoing webhooks work well in conjunction with the IRM API.

For example webhook configurations with common integrations, refer to: