Sift analysis - Slow requests
Overview
This analysis searches for slow traces during the investigation time period.
Inputs
Required:
- A Tempo datasource (This analysis will attempt to discover this datasource without the need for user input)
Optional:
- None
How it works
Step 1: Evaluate any specified services
If services are specified, for each service, search Tempo for traces matching a few conditions where the duration is configurable:
span.http.route != ”/metrics”
span.http.route !~ “/debug/pprof*”
duration > %ms
resource.service.name = “$service”
Step 2: Search without specified services
If services are not specified, search Tempo for traces matching similar conditions, but instead of filtering using service name, convert labels to ‘known’ OTel semantic conventions if possible, and filter using those instead:
span.http.route != ”/metrics”
span.http.route !~ “/debug/pprof*”
duration > %ms
resource.k8s.namespace.name = “$namespace”
resource.k8s.cluster.name = “$cluster”
Step 3: Return results
Return all matching traces.
What resources does this analysis use?
- Tempo’s search API
How does this analysis determine when a result is interesting?
- If there is at least one slow request, the result is considered interesting.
What configurations options are available for this analysis?
Threshold
The threshold above which traces are considered ‘slow’.
- Default: 3 seconds
- Minimum: 1 second