Server-side metrics
Server-side metrics is an experimental feature.
Server-side metrics is a feature that derive metrics from ingested traces.
To generate metrics, it uses an additional component: the metrics-generator. If present, the distributor will write received spans to both the ingester and the metrics-generator. The metrics-generator processes spans and writes metrics to a Prometheus datasource using the Prometheus remote write protocol.
Architecture
Generating and writing metrics introduces a whole new domain to Tempo unlike any other functionality thus far. For this reason, a component, the metrics-generator, is dedicated to working with metrics. This results in a clean division of responsibility and limits the blast radius from a metrics processors or the Prometheus remote write exporter blowing up.
│
│
Ingress
│
▼
┌──────────────────────┐
│ │
│ Distributor │
│ │
└──────────────────────┘
2│ │1
│ │
┌──────────────────┘ └────────┐
│ │
▼ ▼
┌ ─ ─ ─ ─ ─ ─ ─ ─ ┏━━━━━━━━━━━━━━━━━━━━━━┓ ┌──────────────────────┐
│ ┃ ┃ │ │
│ Prometheus ◀────Prometheus ────┃ Metrics-generator ┃ │ Ingester │◀───Queries────
│ Remote Write ┃ ┃ │ │
└ ─ ─ ─ ─ ─ ─ ─ ─ ┗━━━━━━━━━━━━━━━━━━━━━━┛ └──────────────────────┘
│
│
│
▼
┌─────────────────┐
│ │
│ Backend │
│ │
└─────────────────┘
Configuration
For a detailed view of all the config options for the metrics generator, visit its config page
Related Tempo resources
GrafanaCONline 2021
Be the first to learn about exciting next-generation features in Grafana 8.0, be inspired by what community members are building, and attend expert-led sessions and workshops on Grafana, Prometheus, Loki logs, and more.
Getting started with tracing and Grafana Tempo
In this session, we will use an instrumented application to demonstrate how to use logs and Prometheus exemplars to find traces effectively in Tempo. The demo will hit on the basics of operating Tempo and reveal how Tempo allows us to scale tracing as far as possible with less operational cost and complexity than ever before.
Tracing made simple with Grafana
Dive into the new options for viewing tracing data inside Grafana and learn how to make tracing an integral part of your observability strategy.