Adaptive Traces
Overview
In complex systems, where requests traverse multiple services, distributed tracing constructs traces that reveal the path and timing of each request, making them invaluable for pinpointing bottlenecks and interconnection issues. While full tracing provides comprehensive data, healthy applications with consistently successful requests generate a massive volume of unnecessary traces.
Adaptive Traces addresses this by employing tail sampling, a technique that delays the sampling decision until a complete trace is collected. This allows for informed choices, prioritizing critical traces, such as those with errors or high latency, by evaluating the full request context.
With Adaptive Traces, you define policies to control data ingestion, combining rules like ingest traces with ERROR status from service X or capture 5% of traces from service Y. This tailored approach ensures cost-effective debugging of production issues by focusing on the most relevant traces.
Why use Adaptive Traces?
- Manage via Grafana Cloud: handles the complexities of large-scale tail sampling, providing a managed solution with 10-second policy updates, without the operational overhead of a centralized pipeline
- Optimize data: save on ingest costs
- Improve performance: less data needs to be processed, transmitted, and queried
- Reduce toil: customizable rules to ingest only traces of value
