What are traces?

A user on your website enters their email address into a form to sign up for your mailing list. They click Enter.

The user’s email address is data that flows through your system. In a cloud computing world, it is possible that clicking that one button causes data to touch multiple nodes across your cluster of microservices.

The email address may be sent to a verification algorithm sitting in a microservice that exists solely for that purpose. If it passes the check, the information is stored in a database.

Along the way an anonymization node strips personally-identifying data from the address and sends metadata collected to a marketing qualifying algorithm to determine whether the request was sent from a targeted part of the internet.

Services respond and data flows back from each, sometimes triggering new events across the system. Along the way, logs are written in various nodes with a time stamp showing when the info passed through.

Finally, the request and response activity ends and a record of that request is sent to Grafana Cloud.

Grafana Cloud Traces

Grafana Cloud Traces is based on Tempo, an open-source, easy-to-use, and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki. Tempo can be used with any of the open-source tracing protocols, including Jaeger, Zipkin, and OpenTelemetry.

Grafana Cloud Traces lets you search for traces, generate metrics from spans, and link your tracing data with logs and metrics.

Search for traces

Search for traces using common dimensions such as time range, duration, span tags, service names, etc. Use the trace view to quickly diagnose errors and high latency events in your system.

Metrics from spans

RED metrics can be used to drive service graphs and other ready-to-go visualizations of your span data. Metrics generation is disabled by default. Contact Grafana Support to enable metrics generation for your organization.

These metrics exist in your Hosted Metrics instance and can also be easily used to generate powerful custom dashboards.

Metrics automatically generate exemplars as well which allows easy metrics to trace linking. Exemplars are GA in Grafana Cloud so you can also push your own.

Linking traces and logs

If you’re already doing request/response logging with trace IDs, they can be easily extracted from logs to jump directly to your traces.

In the other direction, you can configure Grafana Cloud to create a link from an individual span to your Loki logs. If you see a long-running or errored span, you can immediately jump to the logs of the process causing the error.