Images in notifications
Images in notifications helps recipients of alert notifications better understand why an alert has fired or resolved by including an image of the panel associated with the Grafana managed alert rule.
Note: Images in notifications are not available for Grafana Mimir and Loki managed alert rules, or when Grafana is set up to send alert notifications to an external Alertmanager.
If Grafana is set up to send images in notifications, it takes a screenshot of the panel for the Grafana managed alert rule when either of the following happen:
- The alert rule transitions from pending to firing
- The alert rule transitions from firing to OK
Grafana does not support images for alert rules that are not associated with a panel. An alert rule is associated with a panel when it has both Dashboard UID and Panel ID annotations.
Images are stored in the data path and so Grafana must have write-access to this path. If Grafana cannot write to this path then screenshots cannot be saved to disk and an error will be logged for each failed screenshot attempt. In addition to storing images on disk, Grafana can also store the image in an external image store such as Amazon S3, Azure Blob Storage, Google Cloud Storage and even Grafana where screenshots are stored in
public/img/attachments. Screenshots older than
temp_data_lifetime are deleted from disk but not the external image store. If Grafana is the external image store then screenshots are deleted from
data but not from
Note: It is recommended that you use an external image store, as not all contact points support uploading images from disk. It is also possible that the image on disk is deleted before an alert notification is sent if
temp_data_lifetimeis less than the
group_intervaloptions used in Alertmanager.
To use images in notifications, Grafana must be set up to use image rendering. It is also recommended that Grafana is set up to upload images to an external image store such as Amazon S3, Azure Blob Storage, Google Cloud Storage or even Grafana.
If Grafana has been set up to use image rendering images in notifications can be turned on via the
capture option in
# Enable screenshots in notifications. This option requires the Grafana Image Renderer plugin. # For more information on configuration options, refer to [rendering]. capture = true
It is recommended that
max_concurrent_screenshots is set to a value that is less than or equal to
concurrent_render_request_limit. The default value for both
# The maximum number of screenshots that can be taken at the same time. This option is different from # concurrent_render_request_limit as max_concurrent_screenshots sets the number of concurrent screenshots # that can be taken at the same time for all firing alerts where as concurrent_render_request_limit sets # the total number of concurrent screenshots across all Grafana services. max_concurrent_screenshots = 5
If Grafana has been set up to use an external image store,
upload_external_image_storage should be set to
# Uploads screenshots to the local Grafana server or remote storage such as Azure, S3 and GCS. Please # see [external_image_storage] for further configuration options. If this option is false, screenshots # will be persisted to disk for up to temp_data_lifetime. upload_external_image_storage = false
Restart Grafana for the changes to take effect.
Images in notifications are supported in the following notifiers and additional support will be added in the future:
|Name||Upload images from disk||Include images from URL|
|Google Hangouts Chat||No||Yes|
Include images from URL refers to using the external image store.
Grafana provides the following metrics to observe the performance and failure rate of images in notifications.
For example, if a screenshot could not be taken within the expected time (10 seconds) then the counter
grafana_screenshot_failures_total is updated.
- Images in notifications are not available for Grafana Mimir and Loki managed alert rules, or when Grafana is set up to send alert notifications to an external Alertmanager.
- When alerts generated by different alert rules are sent in a single notification, there may be screenshots for each alert rule. This happens if an alert group contains multiple alerting rules. The order the images are attached is random. If you need to guarantee the ordering of images, make sure that your alert groups contain a single alerting rule.
- Some contact points only handle a single image. In this case, the first image associated with an alert will be attached. Because the ordering is random, this may not always be an image for the same alert rule. If you need to guarantee you receive a screenshot for a particular rule, make sure that your alert groups contain a single alerting rule.
If Grafana has been set up to send images in notifications, however notifications are still being received without images, then check the troubleshooting instructions below.
- Check that Grafana has been set up as per the instructions
- Check that the notifier supports images in notifications as per the table of supported notifiers
- Enable debug logging in Grafana and look for logs with the logger
- If the alert is not associated with a dashboard there will be logs for
Cannot take screenshot for alert rule as it is not associated with a dashboard
- If the alert is associated with a dashboard, but no panel in the dashboard, there will be logs for
Cannot take screenshot for alert rule as it is not associated with a panel
- If images cannot be taken because of mis-configuration or an issue with image rendering there will be logs for
Failed to take an imageincluding the Dashboard UID, Panel ID, and the error message
Related Grafana resources
Unify your data with Grafana plugins: Datadog, Splunk, MongoDB, and more
In this webinar, learn how to leverage Grafana's plugin ecosystem for access to 80+ data sources, including plugins for Datadog, Splunk, MongoDB, and more.
Getting started with Grafana Enterprise and observability
Join the Grafana Labs team for a 30-minute demo of how to get started with the Grafana Stack, so you can go from zero to observability in just a few minutes.