---
title: "Configure TraceQL metrics | Grafana Tempo documentation"
description: "Learn about configuring TraceQL metrics."
---

# Configure TraceQL metrics

> Caution
> 
> TraceQL metrics is an [public preview feature](/docs/release-life-cycle/). Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available TraceQL metrics are enabled by default in Grafana Cloud.

TraceQL language provides metrics queries as an experimental feature. Metric queries extend trace queries by applying a function to trace query results. This powerful feature creates metrics from traces, much in the same way that LogQL metric queries create metrics from logs.

## Before you begin

To use the metrics generated from traces, you need to:

- Set the `local-blocks` processor to active in your `metrics-generator` configuration
- Configure a Tempo data source in Grafana or Grafana Cloud ( [documentation](/docs/grafana/next/datasources/tempo/configure-tempo-data-source/))
- Access Grafana Cloud or Grafana version 10.4 or later

Refer to the [Metrics-generator configuration](/docs/tempo/latest/configuration/#metrics-generator) documentation for more information about the `metrics-generator` configuration.

## Activate and configure the `local-blocks` processor

You must enable the local-blocks processor to start using metrics queries like `{ } | rate()`. If not enabled, then the metrics queries fail with the error `localblocks processor not found`. Enabling the `local-blocks` processor can be done either per tenant or in all tenants.

To activate the `local-blocks` processor for all users, add it to the list of processors in the `overrides` block of your Tempo configuration.

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
# Global overrides configuration.
overrides:
  metrics_generator_processors: ["local-blocks"]
```

To configure the processor per tenant, use the `metrics_generator_processor` override.

Example for per-tenant in the per-tenant overrides:

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
overrides:
  'tenantID':
    metrics_generator_processors:
      - local-blocks
```

By default, for all tenants in the main configuration:

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
overrides:
  defaults:
    metrics_generator:
      processors: [local-blocks]
```

Add this configuration to run TraceQL metrics queries against all spans and not just server spans:

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
metrics_generator:
  processor:
    local_blocks:
      filter_server_spans: false
```

To run metrics queries on historical data, you must configure the local-blocks processor to flush RF1 blocks to object storage:

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
metrics_generator:
  processor:
    local_blocks:
      flush_to_storage: true
```

Setting `flush_to_storage` to `true` ensures that metrics blocks are flushed to storage so TraceQL metrics queries against historical data.

If you configured Tempo using the `tempo-distributed` Helm chart, you can also set `traces_storage` using your `values.yaml` file. Refer to the [Helm chart for an example](https://github.com/grafana/helm-charts/blob/559ecf4a9c9eefac4521454e7a8066778e4eeff7/charts/tempo-distributed/values.yaml#L362).

For more information about overrides, refer to [Standard overrides](/docs/tempo/latest/configuration/#standard-overrides).

### Local blocks and metrics-generator in Azure blob storage and Helm

> Note
> 
> This configuration only applies if you are using a Helm chart, like `tempo-distributed`, to deploy Tempo.

By default, the metrics-generator doesn’t require a backend connection unless you’ve enabled the `local_blocks` processor. The `local_blocks` processor is used for generating metrics from traces, which is required for TraceQL metrics When this configuration is set, the metrics-generator produces blocks and flushes them into a backend storage.

In this case, list the generator in the `env var` expansion configuration so the `STORAGE_ACCOUNT_ACCESS_KEY` has the secret value.

You can use this configuration example with Helm charts, like `tempo-distributed`. Replace any values in all caps with the values for your Helm deployment.

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
generator:
  extraArgs:
    - "-config.expand-env=true"
  extraEnv:
    - name: <STORAGE_ACCOUNT_ACCESS_KEY>
      valueFrom:
        secretKeyRef:
          name: <TEMPO-TRACES-STG-KEY>
          key: <TEMPO-TRACES-KEY>
```

For more information, refer to [Azure hosted storage](/docs/tempo/latest/configuration/hosted-storage/azure/).

## Evaluate query timeouts

Because of their expensive nature, these queries can take a long time to run. As such, consider increasing the timeouts in various places of the system to allow enough time for the data to be returned.

Consider these areas when raising timeouts:

- Any proxy in front of Grafana
- Grafana data source for Prometheus pointing at Tempo
- Tempo configuration
  
  - `querier.search.query_timeout`
  - `server.http_server_read_timeout`
  - `server.http_server_write_timeout`

## Set TraceQL metrics query options

The `query_frontend.metrics` configuration block controls all TraceQL metrics queries. The configuration depends on the environment.

> Note
> 
> The default maximum time range for a metrics query is 3 hours, configured using the `query_frontend.metrics.max_duration` parameter.
> 
> This is different to the default TraceQL maximum time range of 168 hours (7 days).

For example, in a cloud environment, smaller jobs with more concurrency may be desired due to the nature of scale on the backend.

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
query_frontend:
  metrics:
    concurrent_jobs: 1000
    target_bytes_per_job: 2.25e+08 # ~225MB
    interval: 30m0s
```

For an on-prem backend, you can improve query times by lowering the concurrency, while increasing the job size.

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
query_frontend:
  metrics:
    concurrent_jobs: 8
    target_bytes_per_job: 1.25e+09 # ~1.25GB
```

## Sampling and performance optimization

TraceQL metrics queries support sampling hints to improve performance on large datasets. Refer to the [TraceQL metrics sampling](/docs/tempo/latest/metrics-from-traces/metrics-queries/sampling-guide/) documentation for more information.

When using sampling in your TraceQL metrics queries, consider:

- **Timeout settings:** Sampled queries run faster but may still benefit from adequate timeouts
- **Concurrent jobs:** Sampling reduces per-job processing time, allowing higher concurrency