Documentation Grafana Cloud Alerts and IRM IRM Configure Integrations Webhooks Outgoing webhooks

Outgoing webhooks for Grafana IRM

Outgoing webhooks in Grafana IRM enable flexible integration with external systems by automatically sending event data related to alerts and incidents to your preferred tools and APIs.

Webhook presets

Webhook presets provide pre-configured integrations that allow you to quickly set up and customize webhooks to automate actions in your IRM workflow. Each preset is designed for specific use cases and includes optimized default settings that you can further customize based on your requirements.

Available webhook presets

• Simple webhook for alert groups: Sends basic alert group data with minimal configuration.

  Ideal for simple integrations and use in escalation steps where you need straightforward event notifications.

• Advanced webhook for alert groups: Provides full customization capabilities including templating, conditional execution, and advanced filtering.

  Use this preset when you need fine-grained control over webhook behavior and payload formatting.

• Advanced webhook for incidents: Offers comprehensive incident data with customizable settings and templating support.

  Perfect for integrating with external systems that are part of your incident management process or creating custom incident workflows.

• Grafana Assistant for IRM: Forwards alert group or incident data to Grafana Assistant for automated analysis and investigation.

  Use this preset to trigger a Grafana Assistant investigation when an alert group or incident event occurs. Grafana Assistant coordinates multiple specialist agents to analyze incidents in parallel and posts findings to your timeline.

• Grafana Sift for alert groups: Forwards alert group data to Grafana Sift for infrastructure telemetry analysis.

  Use this preset to trigger a Sift investigation when an alert group event occurs. Sift is a free diagnostic assistant that performs infrastructure telemetry investigations, running checks and curating investigative summaries. Investigation results are posted to your alert group timeline as resolution notes.

Configure an outgoing webhook

To create a webhook from the IRM UI:

1. In Grafana Cloud, go to Alerts & IRM > IRM > Integrations.
2. Select the Outgoing Webhooks tab.
3. Click + Create an Outgoing Webhook.
4. Select the preset that applies for your configuration:
   • Simple webhook for alert groups – Sends basic alert group data (used for escalation steps).
   • Advanced webhook for alert groups – Customizable with additional settings and templating.
   • Advanced webhook for incidents – Customizable with additional settings and templating.
   • Grafana Assistant for IRM – Forwards data to Grafana Assistant for automated analysis.
   • Grafana Sift for alert groups – Forwards alert group data to Grafana Sift.
5. Complete the webhook configuration form.
6. Click Create to save the webhook.

For information about configuring outgoing webhooks via Terraform, refer to the Terraform documentation.

Webhook configuration fields

The following fields are available when configuring an outgoing webhook:

Supported triggers

The following event types are available to trigger a webhook to execute.

Alert group events

Incident events

Incident subevents

Use webhook templates

Use Jinja2 templates to include dynamic content from the event context. Templates have access to context data from the triggering event. Supported fields vary by trigger type.

For more information about templates in IRM, refer to the Templates documentation.

Common template fields

• {{ event.type }} – Type of event
• {{ event.time }} – Timestamp
• {{ user.username }} – Username if triggered by a user
• {{ alert_group.title }} – Alert group title
• {{ alert_payload }} – Contents of the first alert in the group
• {{ incident.title }} – Incident title
• {{ incident.severity }} – Incident severity
• {{ webhook.name }} – Webhook name

Example templates

Basic JSON payload:

JSON Copy

{
  "name": "{{ alert_payload.labels.alertname }}",
  "message": "{{ alert_payload.annotations.description }}"
}

URL with query params:

Copy

https://example.com/tickets?assignee={{ user.email }}

Use tojson() to format objects:

JSON Copy

{
  "labels": {{ alert_payload.labels | tojson() }}
}

Reference previous responses

You can access results from previously executed webhooks using the {{ responses }} variable. This enables advanced workflows, such as updating a ticket that was created by an earlier webhook.

Conditional execution

Use the Trigger Template field to control when a webhook executes. The webhook will run only if the template evaluates to an empty string, True, or 1.

Example to trigger only for high-severity alerts:

Copy

{% if alert_payload.labels.severity == "high" %}true{% endif %}

Example to trigger only for incidents with “Critical” severity and the “Customer” label set:

Copy

{% if incident.severity == "Critical" and incident.labels | selectattr('key', 'equalto', 'Customer') | list | length > 0 %}true{% endif %}

View webhook execution results

You can view webhook execution history in the Outgoing Webhooks tab. The list includes the latest events and associated data.

Click a webhook to view detailed execution information:

• The triggered event and context
• Request details: URL, headers, and body
• Response code and response body
• Template evaluation output

You can also find webhook responses associated with specific events:

• For alert group webhooks, a link to the corresponding response appears in the alert group timeline, accessible from the alert group detail page.
• For incident webhooks, execution results appear in the incident timeline under the All activity view.

Retry policy

If a webhook request fails, IRM retries the request up to three times with a one-second interval between attempts. IRM only retries requests if the external system responds with one of the following HTTP error codes:

• 500 - Internal Server Error
• 502 - Bad Gateway
• 503 - Service Unavailable
• 504 - Gateway Timeout

Migrate from Incident outgoing webhooks

Legacy incident webhooks continue to work but will eventually be deprecated in future releases.

To migrate a webhook from the legacy Incident outgoing webhooks app, manually recreate the webhook in the shared IRM Outgoing Webhooks interface with the following settings:

Copy

Name: <your webhook name>
HTTP method: POST
URL: <your URL>
Trigger type: <your original trigger type>
Customize forwarded data [x]
Data template: {{ incident }}

Be aware that the IRM shared webhook requests do not include signature or Cloud Events headers.

After enabling the corresponding IRM webhook, test your new webhook before removing the legacy webhook. Once verified, delete the legacy configuration to avoid duplicate executions.

If your webhook previously used the Incident matches filters trigger, use the Incident changed trigger type. If you want to respond to other incident events, such as declarations or resolutions, create additional webhooks for those trigger types as needed.

Recreate your filter logic using Jinja templating in the Trigger template field. This provides equivalent behavior to the original filtered trigger in the legacy Incident integration.

For example:

-severity:critical is equivalent to {{ incident.severity != "critical" }}

or

and(label:customer severity:critical) can be expressed as {{ incident.severity == "Critical" and incident.labels | selectattr('key', 'equalto', 'Customer') | list | length > 0 }}

Examples and integrations

Outgoing webhooks work well in conjunction with the IRM API.

For example webhook configurations with common integrations, refer to:

• JIRA integration
• ServiceNow integration
• Zendesk integration