This is documentation for the next version of Grafana Tempo documentation. For the latest stable release, go to the latest version.

Documentation Grafana Tempo Set up for tracing

Set up for tracing

Grafana Tempo is an open source and high-scale distributed tracing backend. Tempo lets you search for traces, generate metrics from spans, and link your tracing data with logs and metrics. Tempo also powers Grafana Cloud Traces and Grafana Enterprise Traces.

Distributed tracing visualizes the lifecycle of a request as it passes through a set of applications. For more information about traces, refer to Introduction to traces.

To set up for tracing, you need to:

1. Plan your Tempo deployment to meet your needs. Refer to the Plan your deployment documentation. Check out the examples for ideas on how to get started.
2. Deploy Tempo using the Set up Tempo documentation. Tempo offers different deployment modes. Refer to the Deployment documentation section for more information.
3. Instrument your application or service to emit traces. Use the Set up instrumentation documentation.
4. Set up a collector, like Grafana Alloy or the OpenTelemetry Collector, to offload traces from your application and forward them to Tempo. Refer to the Set up a collector documentation.

Requirements for tracing

Tracing requires client instrumentation, a pipeline for gathering and sending traces, and a backend.

• Client instrumentation enables your application or service to emit traces.
• The pipeline component offloads traces from your application and forwards them to a backend. You can use Grafana Alloy or the OpenTelemetry Collector.
• The backend is the storage and retrieval system for traces, which in this case is Tempo.

Refer to the Tracing pipeline components section for more information about these components.

Tracing pipeline components

To build a tracing pipeline, you need four major components: client instrumentation, pipeline, backend, and visualization.

This diagram illustrates a tracing system configuration:

Client instrumentation

Client instrumentation (1 in the diagram) is the first building block to a functioning distributed tracing visualization pipeline. Client instrumentation is the process of adding instrumentation points in the application that create and offload spans.

Pipeline (Grafana Alloy)

After you instrument your application for tracing, the traces are sent to a backend for storage and visualization. You can build a tracing pipeline that offloads spans from your application, buffers them, and forwards them to a backend. Tracing pipelines are optional since most clients can send directly to Tempo. The pipelines become more critical the larger and more robust your tracing system is.

Grafana Alloy is a service that’s deployed close to the application, either on the same node or within the same cluster (in Kubernetes) to quickly offload traces from the application and forward them to a storage backend. Alloy also abstracts features like trace batching to a remote trace backend store, including retries on write failures.

To learn more about Grafana Alloy and how to set it up for tracing with Tempo, refer to Grafana Alloy configuration for tracing.

Backend (Tempo)

The tracing backend stores and retrieves traces on demand. Grafana Tempo is the distributed tracing backend used to store and query traces.

Visualize tracing data with Grafana

Grafana and Grafana Cloud have a built-in Tempo data source that you can use to query Tempo and visualize traces. For more information, refer to the Tempo data source and the Tempo in Grafana topics.