Configure OnCall incoming webhooks
OnCall incoming webhooks allow external monitoring systems to send alert data directly to Grafana IRM. You can use webhooks to integrate with monitoring tools that don’t have native integrations in IRM.
About OnCall incoming webhooks
OnCall webhooks receive HTTP requests from external systems and convert them into alerts that can be:
- Processed through your defined routing rules
- Grouped with related alerts
- Escalated to the appropriate teams
- Displayed with customized templates
Webhook formats
There are two incoming webhooks integration options:
Webhook – Receives and displays the raw JSON payload exactly as it is sent from the external system.
Formatted Webhook – Parses alert payloads that follow a recognized format, enabling structured alerting and improved usability.
Recognized fields for formatted webhooks
The following fields can be included in a formatted webhook payload:
Field | Description |
---|---|
alert_uid | A unique alert ID for grouping |
title | A title for the alert |
image_url | A URL for an image attached to the alert |
state | Either ok or alerting (helps with auto-resolution) |
link_to_upstream_details | A link back to your monitoring system |
message | Detailed alert information |
While none of these fields are required, including them improves alert handling and display.
Set up a webhook
Create an OnCall incoming webhook integration
- In Grafana IRM, go to the Integrations tab
- Click Monitoring systems and select either Webhook or Formatted Webhook
- Click + New integration
- Provide a name and description for the webhook
- Assign the integration to a team
- Click Create Integration
- Copy the Webhook URL provided on the integration page
Configure the external system
- In your monitoring tool, navigate to the webhook configuration settings
- Paste the Webhook URL from Grafana IRM into the designated field
- Configure the payload format to match Grafana IRM’s expected structure
- Save the configuration and send a test alert
Example webhook payload
To send a test webhook using cURL:
curl -X POST \
https://a-prod-us-central-0.grafana.net/integrations/v1/formatted_webhook/m12xmIjOcgwH74UF8CN4dk0Dh/ \
-H 'Content-Type: application/json' \
-d '{
"alert_uid": "08d6891a-835c-e661-39fa-96b6a9e26552",
"title": "The whole system is down",
"image_url": "https://upload.wikimedia.org/wikipedia/commons/e/ee/Grumpy_Cat_by_Gage_Skidmore.jpg",
"state": "alerting",
"link_to_upstream_details": "https://en.wikipedia.org/wiki/Downtime",
"message": "Smth happened. Oh no!"
}'
Customize alert routing
Once your webhook is set up, customize how alerts are processed:
- Define routing templates – Specify how alerts are categorized
- Set up grouping ID templates – Group related alerts into alert groups
- Modify appearance templates – Adjust how alert notifications appear in UI and messaging apps
- Assign escalation chains – Ensure alerts reach the correct responders
Test and validate
After configuring your webhook integration, it’s important to verify that alerts are being received and processed as expected. There are two ways to test your setup:
Use the demo alert feature
You can use demo alerts to verify basic connectivity. To send a simple demo alert, click Send demo alert from the integration you want to test.
Note
Demo alerts are intended for initial connectivty verification only. They do not:
- Test alert grouping functionality
- Exercise the full ingestion and processing pipeline
- Fully represent the behavior of real incoming webhooks
Test full webhook functionality
For a comprehensive test, including grouping, routing, and template processing, send real HTTP requests to your webhook URL:
- Use a tool like cURL (see the example webhook payload above) to send alerts to your webhook URL.
- Send multiple test alerts with the same
alert_uid
to confirm that grouping works as expected. - Monitor incoming alerts in Grafana IRM to verify correct routing, grouping, and escalation.
- Adjust your integration settings as needed to ensure alerts are processed and escalated according to your requirements.