Powerful new language features for TraceQL
Powerful new language features for TraceQL
We’re excited to roll out three powerful enhancements to TraceQL, giving you more flexibility and performance when querying traces in Grafana Cloud with Tempo.
- Rank your metrics with new
topk(n)andbottomk(n)functions to quickly get your highest and lowest ranking time series. - Aggregate spans over time using
sum_over_time()for built-in cumulative sums, such as total bytes, error counts. - Fetch the latest traces first via the experimental
most_recent=truequery hint.
Ranking topk and bottomk functions for TraceQL metrics
When you’re looking at latency, error rates, or throughput across hundreds or thousands of services or endpoints, it’s easy to get lost in all the data. Previously, you’d have to pull back the full set of aggregates and then manually inspect or post‑process the results to find your worst offenders or best performers.
With topk(n) and bottomk(n), you can immediately narrow your focus to the top‑ or bottom‑ranked spans in a single, efficient query. This saves time and reduces the data volume you need to scan downstream.
{} | avg_over_time(span:duration) by (span:name) | topk(10)
{} | count_over_time() by (span:name) | bottomk(10)These are second stage functions used, where:
topk(n)returns the n series with the highest values from a first‑stage aggregation.bottomk(n)returns the n series with the lowest values.
sum_over_time function for TracesQL metrics
With sum_over_time(), you can directly compute cumulative sums inside TraceQL, like total bytes transferred, total error counts, or resource consumption over time.
{} | sum_over_time(span.http.response_content_length)For each step interval, sum_over_time(attr) totals the values of attr across all matching spans.
Experimental query hint most_recent=true to retrieve the most recent traces
When troubleshooting a live incident or monitoring production health, you often need to see the absolute latest traces first. By default, Tempo’s query engine favors speed and returns the first N matching traces, which may not be the newest.
The most_recent hint ensures you’re always looking at the freshest data, so you can diagnose recent errors or performance regressions without missing anything due to early row‑limit cuts.
{} with (most_recent=true)With most_recent=true, Tempo performs a deeper search across data shards, retains the newest candidates, and returns traces sorted by start time rather than stopping at the first limit hit.
