Grafana Agent Flow mode Tutorials Filter Prometheus metrics
Open source

Filter Prometheus metrics

In this tutorial, you’ll add a new component prometheus.relabel using relabel.river to filter metrics. This tutorial uses the same base as Collecting Prometheus metrics.


Run the example

The prometheus.relabel component is used to drop, add, or filter metrics. Run the following: curl -O && bash ./ relabel.river.

The script does:

  1. Downloads the configs necessary for Mimir, Grafana and the Grafana Agent.
  2. Downloads the docker image for Grafana Agent explicitly.
  3. Runs the docker-compose up command to bring all the services up.

Allow the Grafana Agent to run for two minutes, then navigate to Grafana page and the service label will be there with the api_server value.

Dashboard showing api_server

What’s happening?

  1. The Prometheus scraper is sending the metrics to the filter.
  2. The filter is adding a new label named service with the value api_server.
  3. The filter is then sending metrics to the remote_write endpoint.

Update the service value

Open the relabel.river file that was downloaded and change the name of the service to api_server_v2, then run bash ./ relabel.river. Allow the Grafana Agent to run for two minutes, then navigate to Grafana page, and the new label will be updated. The old value api_server may still show up in the graph but hovering over the lines will show that that value stopped being scraped and was replaced with api_server_v2.

Updated dashboard showing api_server_v2