This is documentation for the next version of Grafana documentation. For the latest stable release, go to the latest version.

Documentation Grafana documentation Data sources OpenTSDB Annotations

OpenTSDB annotations

Annotations allow you to overlay event information on graphs, providing context for metric changes. The OpenTSDB data source supports both metric-specific annotations and global annotations stored in OpenTSDB.

For general information about annotations in Grafana, refer to Annotate visualizations.

Annotation types

OpenTSDB supports two types of annotations:

How Grafana retrieves annotations

When you configure an annotation query, Grafana queries OpenTSDB for the specified metric and retrieves any annotations associated with that metric’s time series. The query includes the globalAnnotations=true parameter, which allows Grafana to also retrieve global annotations when enabled.

Grafana displays the description field from each annotation as the annotation text.

Configure an annotation query

To add OpenTSDB annotations to a dashboard:

1. Navigate to the dashboard you want to update and click Edit.
2. Click the Add new element icon (blue plus sign).
3. Click Annotation query.
4. Enter a name for the annotation query.
5. If you don’t want to use the annotation query right away, clear the Enabled checkbox.
6. Select a color for the annotation event markers.
7. Select an option in the Show annotation controls in drop-down list to control where on the dashboard the annotation is displayed.
8. Select an option in the Show in drop-down list to control the panels in which the annotation is displayed.
9. Click Open query editor to open the Annotation Query dialog box.
10. Select the OpenTSDB data source from the Data source drop-down list.
11. Configure the annotation query and field mappings.
12. (Optional) Click Test annotation query to ensure that the query is working properly.
13. Click Close when you’ve completed the query setup.
14. Click Save.
15. (Optional) Enter a description of the changes you’ve made.
16. Click Save.
17. Click Exit edit.

Annotation query fields

Example annotation queries

The following examples demonstrate common annotation use cases.

Track application deployments

Monitor when deployments occur for a specific application:

This query retrieves annotations attached to the deploy.myapp metric, showing deployment events for that specific application.

Monitor infrastructure-wide events

Capture system-wide events such as network changes or datacenter maintenance:

This query retrieves both metric-specific and global annotations, providing a complete picture of infrastructure events.

Track incidents and outages

Mark incident start and resolution times:

Monitor configuration changes

Track when configuration changes are applied:

Correlate multiple event types

You can add multiple annotation queries to a single dashboard to correlate different event types. For example:

1. Add a “Deployments” annotation query for deploy.* metrics.
2. Add an “Incidents” annotation query for events.incident.
3. Add a “Maintenance” annotation query with global annotations enabled.

This allows you to see how deployments, incidents, and maintenance windows relate to your metric data.

How annotations appear

Annotations appear as vertical lines on time series panels at the timestamps where events occurred. Hover over an annotation marker to view:

• The annotation name (from your query configuration)
• The event description (from the OpenTSDB annotation’s description field)
• The timestamp

Different annotation queries can be assigned different colors in the annotation settings to distinguish between event types.

Create annotations in OpenTSDB

To display annotations in Grafana, you must first create them in OpenTSDB. OpenTSDB provides an HTTP API for managing annotations.

Annotation data structure

OpenTSDB annotations have the following fields:

Create a global annotation

Use the OpenTSDB API to create a global annotation:

sh Copy

curl -X POST http://<OPENTSDB_HOST>:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d '{
    "startTime": 1609459200,
    "description": "Production deployment v2.5.0",
    "notes": "Deployed new feature flags and performance improvements",
    "custom": {
      "version": "2.5.0",
      "environment": "production",
      "deployer": "jenkins"
    }
  }'

Create a metric-specific annotation

To attach an annotation to a specific time series, include the tsuid:

sh Copy

curl -X POST http://<OPENTSDB_HOST>:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d '{
    "startTime": 1609459200,
    "endTime": 1609462800,
    "tsuid": "000001000001000001",
    "description": "Server maintenance",
    "notes": "Scheduled maintenance window for hardware upgrade"
  }'

To find the TSUID for a metric, use the OpenTSDB /api/uid/tsmeta endpoint.

Create annotations programmatically

Integrate annotation creation into your deployment pipelines or monitoring systems:

Deployment script example:

sh Copy

#!/bin/bash
VERSION=$1
TIMESTAMP=$(date +%s)

curl -X POST http://opentsdb.example.com:4242/api/annotation \
  -H "Content-Type: application/json" \
  -d "{
    \"startTime\": $TIMESTAMP,
    \"description\": \"Deployed version $VERSION\",
    \"custom\": {
      \"version\": \"$VERSION\",
      \"environment\": \"production\"
    }
  }"

For more details on the annotation API, refer to the OpenTSDB annotation API documentation.

Troubleshoot annotation issues

The following section addresses common issues you may encounter when using OpenTSDB annotations.

Annotations don’t appear

Possible causes and solutions:

Annotation text is empty

The annotation displays but has no description text.

Solution: Ensure the description field is populated when creating annotations in OpenTSDB. Grafana displays the description field as the annotation text.

Next steps

• Build queries to visualize metrics alongside annotations.
• Use template variables to create dynamic dashboards.
• Set up alerting to get notified when metrics cross thresholds.