Menu
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Cost management and billingbreadcrumb arrow Adaptive Telemetrybreadcrumb arrow Adaptive Tracesbreadcrumb arrow Guidesbreadcrumb arrow Drop low-value traces to reduce noise
Grafana Cloud

Drop low-value traces to reduce noise

In this scenario, you create drop policies to filter out and discard low-value traces at the point of ingestion.

Goal

Imagine multiple services in your environment use a standard /health-check endpoint that is polled every second by load balancers. This can generate millions of high-volume, low-value traces that add noise and increase costs without providing real troubleshooting value.

Your goal is to drop all of these health check traces globally, regardless of which service they come from.

To do this, you need a way to:

  • Identify and target traces from a specific endpoint, regardless of the service.
  • Drop these traces at the point of ingestion to reduce noise and cost across the board.
  • Ensure that all other traces from all services are unaffected and continue to be processed normally.

Solution

To achieve this, you can create a single, specific drop policy. This policy will match and filter out any trace that has an http.route attribute of /health-check.

Note

When you create a drop policy, make sure you also have an active sampling policy (like always_sample) otherwise all of your traces will be dropped.

Here is an example using JSON.

// Policy: Drop /health-check traces from all services
{
  "drop_sub_policy": [
    {
      "name": "uninteresting-endpoints",
      "type": "string_attribute",
      "string_attribute": {
        "key": "http.route",
        "values": [
          "/health-check"
        ]
      }
    }
  ]
}

Note

Drop policies are evaluated before all other policies and take precedence. If a trace matches a drop policy you’ve created, it is not evaluated by any of your other policies.

Be sure to scope your drop policies carefully. In this case, the policy is intentionally broad to cover all services.

Outcome

With this policy active, you have successfully filtered out the noise globally.

  • Traces from any service’s /health-check endpoint are now dropped upon ingestion.
  • All your other valuable traces from all services are processed as normal, subject to any other keep policies.
  • Your team’s dashboards and troubleshooting views are now cleaner, making it easier to spot real issues across the entire platform.