Catch pipeline issues across your services
Catch pipeline issues across your services
The entity catalog — and the Application page, which shows that same catalog scoped to services — now has a Pipeline issues tab, next to Instrumentation quality, that lists detected telemetry-pipeline problems for your whole stack in one place. Instead of opening each service to check whether its data looks right, you can check the pipeline that all of your services send through at once.
See what needs attention, at a glance
The Pipeline issues tab carries a count of open warning and critical issues, so you can spot problems across your whole stack without opening anything. The count disappears when there’s nothing to report.
What it checks
Pipeline issues currently looks for five problems:
- Trace metrics have hit their series limit. The active-series limit for metrics generated from traces has been reached, so some series are being dropped.
- A metric’s labels have hit their limit. The labels-per-series limit has been exceeded, so some series are being dropped.
- A trace was too large to ingest. Some traces can’t be ingested because they’re too large.
- You’re being rate limited. The ingestion rate limit has been exceeded.
- Spans are arriving too late. Spans are showing up outside the metrics-generation ingestion window, so some metrics can’t be generated from them.
All five checks depend on reading your Grafana Cloud usage data and resolving your stack’s identifiers. When that information isn’t fully available, the tab shows a distinct notice that it couldn’t fully check the pipeline, rather than reporting the pipeline as healthy.
Go straight to a fix
Each issue comes with a Contact support action, so you can get help resolving it.
Availability
Pipeline issues is available in private preview and ships off by default. Contact your Grafana Labs representative to have it enabled for your organization.
