TraceQL

This is documentation for the next version of Tempo. For the latest stable release, go to the latest version.

Open source RSS

TraceQL

Inspired by PromQL and LogQL, TraceQL is a query language designed for selecting traces in Tempo. Currently, TraceQL query can select traces based on the following:

  • Span and resource attributes, timing, and duration
  • Basic aggregates: count(), avg(), min(), max(), and sum()

Read the blog post, Get to know TraceQL, for an introduction to TraceQL and its capabilities.

There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, check out our blog!

The TraceQL language uses similar syntax and semantics as PromQL and LogQL, where possible.

Check the Tempo release notes for the latest updates to TraceQL.

Requirements

TraceQL requires the Parquet columnar format, which is enabled by default. For information on Parquet, refer to the Apache Parquet backend documentation.

Use TraceQL

To use TraceQL, you need to:

Query using TraceQL

You can use TraceQL query editor and query builder in the Tempo data source to build queries and drill-down into result sets. The editor and builder are available in the Tempo data source for Grafana Explore.

Query editor showing request for http.method

In addition, you can use Traces Drilldown to investigate your tracing data without writing TraceQL queries. For more information, refer to the Traces Drilldown documentation.

For more information, refer to Write TraceQL queries in Grafana.

Stream query results

Query results can stream to the client, which lets you look at traces matching your query before the entire query completes.

The GRPC streaming API endpoint in the query frontend allows a client to stream search results from Tempo. The tempo-cli also uses this streaming endpoint. For more information, refer to the Tempo CLI documentation.

To use streaming in Grafana, you must have stream_over_http_enabled: true enabled in Tempo. For information, refer to Tempo GRPC API.

TraceQL metrics

Caution

TraceQL metrics is an experimental feature. Engineering and on-call support is not available. Documentation is either limited or not provided outside of code comments. No SLA is provided. Contact Grafana Support to enable this feature in Grafana Cloud.

TraceQL metrics are easy to get started with. Refer to TraceQL metrics for more information.

You can also use TraceQL metrics queries. Refer to TraceQL metrics queries for more information.