Get started with Grafana Tempo
You can use Grafana Cloud to avoid installing, maintaining, and scaling your own instance of Grafana Tempo. Create a free account to get started, which includes free forever access to 10k metrics, 50GB logs, 50GB traces, & more.
Distributed tracing visualizes the lifecycle of a request as it passes through a set of applications. For more information about traces, refer to What are traces?.
Grafana Tempo is an open-source, easy-to-use, and high-scale distributed tracing backend. Tempo lets you search for traces, generate metrics from spans, and link your tracing data with logs and metrics.
To build a tracing pipeline, you need four major components: client instrumentation, pipeline, backend, and visualization.
This diagram illustrates a tracing system configuration:
Client instrumentation
Client instrumentation (1 in the diagram) is the first building block to a functioning distributed tracing visualization pipeline. Client instrumentation is the process of adding instrumentation points in the application that create and offload spans.
Most of the popular client instrumentation frameworks have SDKs in the most commonly used programming languages. You should pick one according to your application needs.
OpenTelemetry auto-instrumentation
Some languages have support for auto-instrumentation. These libraries capture telemetry information from a client application with minimal manual instrumentation of the codebase.
- OpenTelemetry Java auto-instrumentation
- OpenTelemetry .NET auto-instrumentation
- OpenTelemetry Python auto-instrumentation
Note: Check out the instrumentation references to learn how to instrument your favorite language for distributed tracing.
Pipeline (Grafana Agent)
Once your application is instrumented for tracing, the traces need to be sent to a backend for storage and visualization. You can build a tracing pipeline that offloads spans from your application, buffers them, and eventually forwards them to a backend. Tracing pipelines are optional (most clients can send directly to Tempo), but the pipelines become more critical the larger and more robust your tracing system is.
The Grafana Agent is a service that is deployed close to the application, either on the same node or within the same cluster (in Kubernetes) to quickly offload traces from the application and forward them to a storage backend. The Grafana Agent also abstracts features like trace batching to a remote trace backend store, including retries on write failures.
To learn more about the Grafana Agent and how to set it up for tracing with Tempo, refer to this blog post.
Note: The OpenTelemetry Collector / Jaeger Agent can also be used at the agent layer. Refer to this blog post to see how the OpenTelemetry Collector can be used with Grafana Cloud Tempo.
Backend (Tempo)
Grafana Tempo is an easy-to-use and high-scale distributed tracing backend used to store and query traces. The tracing backend stores and retrieves traces on demand.
Getting started with Tempo is easy.
First, check out the examples for ideas on how to get started with Tempo.
Next, review the Setup documentation for step-by-step instructions for setting up a Tempo cluster and creating a test application.
For production workloads, refer to the deployment section.
Note: The Grafana Agent is already set up to use Tempo. Refer to the configuration and example for details.
Visualization (Grafana)
Grafana has a built-in Tempo data source that can be used to query Tempo and visualize traces. For more information, refer to the Tempo data source and the Tempo in Grafana topics.
For details about Grafana configuration, refer to querying configuration.
For more information, refer to the Tempo in Grafana documentation.
