This is documentation for the next version of Grafana documentation. For the latest stable release, go to the latest version.

Documentationbreadcrumb arrow Grafana documentationbreadcrumb arrow Data sourcesbreadcrumb arrow Tempo
Grafana Cloud Enterprise Open source

Tempo data source

Grafana ships with built-in support for Grafana Tempo, a high-volume, minimal-dependency distributed tracing backend from Grafana Labs. Use the Tempo data source to search and visualize traces, correlate traces with logs, metrics, and profiles, and monitor service dependencies with the Service Graph.

Want to learn more about traces and the other telemetry signals? Refer to Understand your data.

Note

Grafana Cloud users: Grafana Cloud includes Grafana Cloud Traces, a pre-configured tracing data source backed by Tempo. You can use Grafana Cloud Traces to query traces without additional setup. Use the Tempo data source when you need to connect to a self-managed Tempo instance or require custom configuration such as trace correlations.

Supported features

The Tempo data source supports the following features:

FeatureSupportedNotes
TraceQL queriesYesQuery traces using TraceQL, the query language designed for traces
SearchYesFind traces by service name, span name, duration, and attributes
Service GraphYesVisualize service dependencies and RED metrics
Trace to logsYesNavigate from spans to related logs in Loki and other log data sources
Trace to metricsYesLink spans to metrics queries in Prometheus
Trace to profilesYesLink spans to profiling data in Grafana Pyroscope
Trace correlationsYesEmbed custom correlation links in trace views
StreamingYesDisplay TraceQL results as they become available
JSON trace uploadYesUpload and visualize trace files without a Tempo instance
ExploreYesAd-hoc trace investigation without dashboards
AlertingNoUse TraceQL metrics in Prometheus for trace-based alerting

Tip

New to tracing? Learn what telemetry signals are and how they work together in Understand your data (Grafana Cloud), or read the Introduction to tracing for core concepts like spans, traces, and instrumentation.

Prefer a queryless experience? Grafana Traces Drilldown lets you explore tracing data using RED metrics without writing TraceQL queries.

Get started

The following pages help you set up and use the Tempo data source:

Connect traces to other signals

After you’ve connected Grafana to Tempo, you can configure correlations between traces and other signals:

  • Trace to logs: Navigate from spans to related logs in Loki, including bidirectional linking.
  • Trace to metrics: Link spans to metrics queries in Prometheus or other metrics data sources.
  • Trace to profiles: Link spans to profiling data in Grafana Pyroscope with embedded flame graphs.
  • Trace correlations: Create custom correlation links to any data source or external URL.

Query with the Grafana CLI

You can query this data source from the command line and from AI coding agents using the Grafana CLI, gcx. gcx gives you and your agent structured, terminal-based access to your Grafana data sources, which is useful for automation and agent-driven investigations.

gcx supports Grafana Cloud and Grafana OSS or Enterprise v12 and later. Before you query, install and authenticate gcx. Refer to the gcx CLI documentation for installation, configuration, and the full command reference.

For example, to search and query your Tempo data source, use the gcx datasources tempo commands:

sh 
# Search for traces with a TraceQL query
gcx datasources tempo query -d <DATASOURCE_UID> '{ span.http.status_code >= 500 }' --since 1h

# Retrieve a single trace by its ID
gcx datasources tempo get -d <DATASOURCE_UID> <TRACE_ID>

# List all trace labels, or the values for a single label
gcx datasources tempo labels -d <DATASOURCE_UID>
gcx datasources tempo labels -d <DATASOURCE_UID> --label service.name

# Run a TraceQL metrics query that returns time series
gcx datasources tempo metrics -d <DATASOURCE_UID> '{ } | rate()' --since 1h

Replace <DATASOURCE_UID> with the UID of your Tempo data source and <TRACE_ID> with a hex-encoded trace ID. You can omit the -d flag when datasources.tempo is configured in your gcx context. The query command searches for traces, get retrieves a single trace by ID, and metrics returns time series from a TraceQL metrics query.