Menu

Caution

Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.

Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Open source

Filter Prometheus metrics

In this tutorial, you’ll add a new component prometheus.relabel using relabel.river to filter metrics. This tutorial uses the same base as Collecting Prometheus metrics.

Prerequisites

Run the example

The prometheus.relabel component is used to drop, add, or filter metrics.

Run the following:

bash
curl https://raw.githubusercontent.com/grafana/agent/main/docs/sources/flow/tutorials/assets/runt.sh -O && bash ./runt.sh relabel.river

The runt.sh script does:

  1. Downloads the configs necessary for Mimir, Grafana and Grafana Agent.
  2. Downloads the docker image for Grafana Agent explicitly.
  3. Runs the docker-compose up command to bring all the services up.

Allow Grafana Agent to run for two minutes, then navigate to Grafana page and the service label will be there with the api_server value.

Dashboard showing api_server

What’s happening?

  1. The Prometheus scraper is sending the metrics to the filter.
  2. The filter is adding a new label named service with the value api_server.
  3. The filter is then sending metrics to the remote_write endpoint.

Update the service value

Open the relabel.river file that was downloaded and change the name of the service to api_server_v2, then run bash ./runt.sh relabel.river. Allow the Grafana Agent to run for two minutes, then navigate to Grafana page, and the new label will be updated. The old value api_server may still show up in the graph but hovering over the lines will show that that value stopped being scraped and was replaced with api_server_v2.

Updated dashboard showing api_server_v2