Documentation Grafana Cloud Alerts and IRM Alerting Configure alert rules Template annotations and labels Template reference

Annotation and label template reference

Annotations and labels in alert rules can be defined using plain text. However, you can also define templates to customize their values with dynamic data from alert rule queries.

For example, you can template the summary annotation to include information from query values, providing relevant alert context for responders. Refer to Template annotations and labels for various use cases.

In templates, variables represent dynamic values from queries, while functions perform actions to transform or format this data.

Variables

Variables represent dynamic values from alert rule queries that can be displayed or accessed in your templates.

The $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation in alert rule templates.

Copy

{{ $values.A.Value }}

Templates are based on the Go templating system. Refer to Template language for additional information.

The following variables are available when templating annotations and labels:

$labels

The $labels variable contains all labels from the query. It excludes user-configured and reserved labels, containing only query labels.

For example, suppose you have a query that returns CPU usage for all of your servers, and you have an alert rule that fires when any of your servers have exceeded 80% CPU usage for the last 5 minutes. You want to add a summary annotation to the alert that tells you which server is experiencing high CPU usage. With the $labels variable you can write a template that prints a human-readable sentence such as:

Copy

CPU usage for {{ $labels.instance }} has exceeded 80% for the last 5 minutes

The outcome of this template would be:

Copy

CPU usage for server1 has exceeded 80% for the last 5 minutes

If you are using a classic condition then $labels will not contain any labels from the query. Classic conditions discard these labels in order to enforce uni-dimensional behavior (at most one alert per alert rule). If you want to use labels from the query in your template then use the example here.

$values

The $values variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (e.g, A, B, C, etc.). It does not contain the results of range queries, as they can return hundreds or thousands of rows.

Each Ref IDs, such as $values.A, has the following properties

Here’s the previous example printing now the value of the instant query with Ref ID A:

Copy

{{ $values.A.Value }} CPU usage for {{ $labels.instance }} over the last 5 minutes.

If the alert has the label instance=server1 and the query returns 81.2345, the template would print:

Copy

81.2345 CPU usage for instance1 over the last 5 minutes.

If the query in Ref ID A is a range query rather than an instant query then add a reduce expression with Ref ID B and replace $values.A.Value with $values.B.Value:

Copy

{{ $values.B.Value }} CPU usage for {{ $labels.instance }} over the last 5 minutes.

Alternatively, you can use the index() function to retrieve the query value:

Copy

{{ index $values "B" }} CPU usage for {{ index $labels "instance" }} over the last 5 minutes.

$value

The $value variable is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule.

This example prints the $value variable:

Copy

{{ $value }}: CPU usage has exceeded 80% for the last 5 minutes.

It would display something like this:

Copy

[ var='A' labels={instance=instance1} value=81.234 ]: CPU usage has exceeded 80% for the last 5 minutes.

Instead, we recommend using $values, which contains the same information as $value but is structured in an easier-to-use table format.

Functions

Functions can perform actions in templates such as transforming or formatting data.

Note that the functions provided by Go’s template language, such as index, and, printf, and len, are available, along with many others.

In addition, the following functions are also available for templating annotations and labels:

Numbers

Strings

Others

For further context on these functions, note that templating in Grafana is based on the Prometheus template implementation, enabling the use of these functions and Prometheus-like templates for formatting alert messages within Grafana.

humanize

The humanize function humanizes decimal numbers:

Copy

{{ humanize 1000.0 }}

Copy

1k

humanize1024

The humanize1024 works similar to humanize but but uses 1024 as the base rather than 1000:

Copy

{{ humanize1024 1024.0 }}

Copy

1ki

humanizeDuration

The humanizeDuration function humanizes a duration in seconds:

Copy

{{ humanizeDuration 60.0 }}

Copy

1m 0s

humanizePercentage

The humanizePercentage function humanizes a ratio value between 0 and 1 to a percentage:

Copy

{{ humanizePercentage 0.2 }}

Copy

20%

humanizeTimestamp

The humanizeTimestamp function humanizes a Unix timestamp:

Copy

{{ humanizeTimestamp 1577836800.0 }}

Copy

2020-01-01 00:00:00 +0000 UTC

toTime

The toTime function converts a Unix timestamp in seconds to time.:

Copy

{{ toTime 1727802106 }}

Copy

2024-10-01 17:01:46 +0000 UTC

title

The title function capitalizes the first character of each word:

Copy

{{ title "hello, world!" }}

Copy

Hello, World!

toUpper

The toUpper function returns all text in uppercase:

Copy

{{ toUpper "Hello, world!" }}

Copy

HELLO, WORLD!

toLower

The toLower function returns all text in lowercase:

Copy

{{ toLower "Hello, world!" }}

Copy

hello, world!

stripPort

The stripPort splits string into host and port, then returns only host:

Copy

{{ stripPort "example.com:8080" }}

Copy

example.com

match

The match function matches the text against a regular expression pattern:

Copy

{{ match "a.*" "abc" }}

Copy

true

reReplaceAll

The reReplaceAll function replaces text matching the regular expression:

Copy

{{ reReplaceAll "localhost:(.*)" "example.com:$1" "localhost:8080" }}

Copy

example.com:8080

graphLink

The graphLink function returns the path to the graphical view in Explore for the given expression and data source:

Copy

{{ graphLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}

Copy

/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":false,"range":true}]

parseDuration

The parseDuration function parses a duration string such as “1h” into the number of seconds it represents.

Copy

{{ parseDuration "1h" }}

Copy

3600

stripDomain

The stripDomain removes the domain part of a FQDN, leaving port untouched:

Copy

{{ stripDomain "example.com:8080" }}

Copy

example:8080

tableLink

The tableLink function returns the path to the tabular view in Explore for the given expression and data source:

Copy

{{ tableLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}

Copy

/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":true,"range":false}]

args

The args function translates a list of objects to a map with keys arg0, arg1 etc. This is intended to allow multiple arguments to be passed to templates:

Copy

{{define "x"}}{{.arg0}} {{.arg1}}{{end}}{{template "x" (args 1 "2")}}

Copy

1 2

safeHtml

The safeHtml function marks string as HTML not requiring auto-escaping:

Copy

{{ safeHtml "<b>Text</b>"}}

Copy

<b>Text</b>

externalURL

The externalURL function returns the external URL of the Grafana server as configured in the ini file(s):

Copy

{{ externalURL }}

Copy

https://example.com/grafana

pathPrefix

The pathPrefix function returns the path of the Grafana server as configured in the ini file(s):

Copy

{{ pathPrefix }}

Copy

/grafana

Differences with notification templates

Both notification templates and alert rule templates use the Go templating system. However, the functions and variables available in notification templates differ from those used in annotations and labels templates, which are described in this documentation.

Annotation and label templates operate in the context of an individual alert instance, while notification templates apply to a notification that includes a group of alert(s).

For example, notification templates provide the .Alerts variable, which includes the list of all firing and resolved alerts in the notification. This variable is not available in alert rule templates, which operate within the context of a single alert instance.

Additionally, you cannot reuse templates for labels and annotations as you can with notification templates. Instead, you need to write each template inline within the label or annotation fields and manually copy them wherever you want to reuse the templates.