Configure Grafana Cloud Asserts notifications

This topic describes how to configure Asserts notifications so that you can link back to Asserts RCA workbench from SLO and Asserts notification alerts.

For more information about how Asserts notification alerts work, refer to How notifications work in Grafana Cloud Asserts.

Note
Asserts notifications are only supported with the Prometheus Alertmanager and not the Grafana Alertmanager.

Before you begin

Before you configure Grafana Cloud Asserts notifications, ensure that you have configured a contact point.

Configure SLO alerts

If you are managing SLOs from Asserts, you must create a notification policy and configure an Alertmanager template.

Create a notification policy

To create a notification policy, complete the following steps:

In the left-side menu, click Alerts & IRM > Alerting > Notification policies.
From the Choose Alertmanager drop-down menu, select the Prometheus Alertmanager.
By default, the Grafana Alertmanager is selected.
Click +New child policy from the default policy.
In the Matching labels section, add the following labels and values:
Label Operator Value
grafana_slo_provenance = asserts
grafana_slo_severity = critical
In the Contact point drop-down menu, select the contact point to send notifications to.
Select the Override grouping toggle.
Select or enter the following Group by values:
- alertname
- asserts_env
- namespace
Click Save policy.

Label	Operator	Value
`grafana_slo_provenance`	`=`	`asserts`
`grafana_slo_severity`	`=`	`critical`

Configure Alertmanager template

If Asserts manages your SLO, you can link back to RCA workbench by either modifying an existing Alertmanager template or using a pre-installed Asserts Alertmanager template.

Modify Alertmanager template

You can modify your existing Alertmanager template to include a URL that links back to Asserts RCA workbench.

The following code snippet shows how to modify a Slack template:

{{- if eq .CommonLabels.grafana_slo_provenance "asserts" }}
  :fire: *<INSTANCE_URL/a/grafana-asserts-app/assertions?env[0]={{ .CommonLabels.asserts_env }}{{ with .CommonLabels.asserts_site }}&site[0]={{ . }}{{ end }}&grafana_slo_uuid={{ .CommonLabels.grafana_slo_uuid }}&start={{(index .Alerts 0).StartsAt.Unix}}000-30m&end={{(index .Alerts 0).StartsAt.Unix}}000%2B30m|Asserts RCA Workbench>*
{{- end }}

Where INSTANCE_URL is the URL of the instance Asserts is running, for example:

https://acme.grafana.net

Use a pre-installed template

After you’ve created a notification policy and contact point, you can associate that contact point with a pre-installed Alertmanager template. These templates are named <INTEGRATION TYPE>.asserts.<TITLE|TEXT>, for example, slack.asserts.text.

The following image shows some installed templates.

After you associate a pre-installed Alertmanager template with a contact point, you receive notifications that contain information about the alert and a link to troubleshoot the alert in Asserts.

Configure Asserts notification alerts

Asserts also lets you to create notifications from assertions. For more information about how Asserts notification alerts work, refer to How notifications work in Grafana Cloud Asserts.

To configure an Asserts notification alert, you must create a notification policy and configure an Alertmanager template.

Create a notification policy

To create a notification policy, complete the following steps:

In the left-side menu, click Alerts & IRM > Alerting > Notification policies.
From the Choose Alertmanager drop-down menu, select the Prometheus Alertmanager.
By default, the Grafana Alertmanager is selected.
Click +New child policy from the default policy.
In the Matching labels section, add the following labels and values:
Label Operator Value
asserts_alert_category =~ .+
alertname = NOTIFICATION_ALERTS
In the Contact point drop-down, select the contact point to send notifications to.
Select the Override grouping toggle.
Select or enter the following Group by values:
- alertname
- asserts_env
- asserts_notification_rule_name
- namespace
- source_alertname
Click Save policy.

Label	Operator	Value
`asserts_alert_category`	`=~`	`.+`
`alertname`	`=`	`NOTIFICATION_ALERTS`

Configure Alertmanager template

When using Asserts notifications, you can link back to RCA workbench by either modifying an existing Alertmanager template or using a pre-installed Asserts Alertmanager template.

Modify Alertmanager template

You can modify your existing Alertmanager template to include a URL that links back to Asserts RCA workbench.

The following code snippet shows how to modify a Slack template:

{{- if eq .GroupLabels.alertname "NOTIFICATION_ALERTS" }}
  :fire: *<INSTANCE_URL/a/grafana-asserts-app/assertions?env[0]={{ .CommonLabels.asserts_env }}{{ with .CommonLabels.asserts_site }}&site[0]={{ . }}{{ end }}&start={{(index .Alerts 0).StartsAt.Unix}}000-30m&end={{(index .Alerts 0).StartsAt.Unix}}000%2B30m&search=Show%20Assertion%20{{ .CommonLabels.asserts_assertion_name }}|Asserts RCA Workbench>*
{{- end }}

Where INSTANCE_URL is the URL of the instance Asserts is running, for example:

https://acme.grafana.net

Use a pre-installed template

After you’ve created a notification policy and contact point, you can associate that contact point with a pre-installed Alertmanager template. These templates are named <integration-type>.asserts.<title|text>, for example, slack.asserts.text.

The following image shows some installed templates.

After you associate a pre-installed Alertmanager template with a contact point, you receive notifications that contains information about the alert and a link to troubleshoot the alert in Asserts.