Documentation Grafana Agent Flow mode Tutorials Chain Prometheus components

Chain Prometheus components

This tutorial shows how to use multiple-inputs.river to send data to several different locations. This tutorial uses the same base as Filtering metrics.

A new concept introduced in Flow is chaining components together in a composable pipeline. This promotes the reusability of components while offering flexibility.

Prerequisites

• Docker

Run the example

Run the following

bash Copy

curl https://raw.githubusercontent.com/grafana/agent/main/docs/sources/flow/tutorials/assets/runt.sh -O && bash ./runt.sh multiple-inputs.river

The runt.sh script does:

1. Downloads the configurations necessary for Mimir, Grafana, and Grafana Agent.
2. Downloads the docker image for Grafana Agent explicitly.
3. Runs the docker-compose up command to bring all the services up.

Allow Grafana Agent to run for two minutes, then navigate to Grafana to see Grafana Agent scrape metrics. The node_exporter metrics also show up now.

There are two scrapes each sending metrics to one filter. Note the job label lists the full name of the scrape component.

Multiple outputs

river Copy

prometheus.scrape "agent" {
    targets    = [{"__address__" = "localhost:12345"}]
    forward_to = [prometheus.relabel.service.receiver]
}

prometheus.exporter.unix "default" {
    set_collectors = ["cpu", "diskstats"]
}

prometheus.scrape "unix" {
    targets    = prometheus.exporter.unix.default.targets
    forward_to = [prometheus.relabel.service.receiver]
}

prometheus.relabel "service" {
    rule {
        source_labels = ["__name__"]
        regex         = "(.+)"
        replacement   = "api_server"
        target_label  = "service"
    }
    forward_to = [prometheus.remote_write.prom.receiver]
}

prometheus.remote_write "prom" {
    endpoint {
        url = "http://mimir:9009/api/v1/push"
    }
}

In the Flow block, prometheus.relabel.service is being forwarded metrics from two sources prometheus.scrape.agent and prometheus.exporter.unix.default. This allows for a single relabel component to be used with any number of inputs.

Adding another relabel

In multiple-input.river add a new prometheus.relabel component that adds a version label with the value of v2 to all metrics after the prometheus.relabel.service.