Tempo in Grafana
Grafana has a built-in Tempo datasource that can be used to query Tempo and visualize traces. This page describes the high-level features and their availability. Use the latest versions for best compatibility and stability.
View trace by ID
The most basic functionality is to visualize a trace using its ID. Select the Trace ID tab and enter the ID to view it. This functionality is enabled by default and is available in all versions of Grafana.
Traces can be discovered by searching logs for entries containing trace IDs. This is most useful when your application also logs relevant information about the trace that can also be searched, such as HTTP status code, customer ID, etc. This feature requires Grafana 7.5 or later, with a linked Loki data source, and a traceID derived field.
Tempo includes the ability to search recent traces held in ingesters. Traces can be searched for data originating from a specific service, duration range, span, or process-level attributes included in your application’s instrumentation, such as HTTP status code and customer ID.
Search of recent traces
Search of recent traces is disabled by default. Ingesters default to storing the last 15 minutes of traces.
To enable recent traces search:
- Run Tempo, enabling search in the YAML configuration. Refer to the search configuration documentation.
- Run Grafana 8.2 or a more recent version. Enable the
Search of the backend datastore
Tempo includes the the ability to search the entire backend datastore.
To enable search of the backend datastore:
- Run Tempo, enabling search in the YAML configuration. Refer to the search configuration documentation. Further configuration information is in backend search. The Tempo configuration is the same for searching recent traces or for search of the backend datastore.
- Run Grafana 8.3.6 or a more recent version. Enable the
tempoBackendSearchfeature toggle. This will cause Grafana to pass the
endparameters necessary for the backend datastore search.
Grafana provides a built-in Application Performance Management (APM) dashboard available in Grafana Cloud and Grafana 9.1. The Grafana APM dashboard visualizes the span metrics (traces data for rates, error rates, and durations (RED)) and service graphs. Once the requirements are set up, this pre-configured dashboard is immediately available in Explore > Service Graphs.
For more information, refer to the APM dashboard documentation.
View JSON file
A local JSON file containing a trace can be uploaded and viewed in the Grafana UI. This is useful in cases where access to the original Tempo datasource is limited, or for preserving traces outside of Tempo. The JSON data can be downloaded via the Tempo API or the Inspector panel while viewing the trace in Grafana.
Related Tempo resources
Be the first to learn about exciting next-generation features in Grafana 8.0, be inspired by what community members are building, and attend expert-led sessions and workshops on Grafana, Prometheus, Loki logs, and more.