Slide 2 of 14

Configuration pipelines

Configuration pipelines

A configuration pipeline is the unit you create in Fleet Management. When you configure a collector remotely, you’re creating at least one pipeline. Each pipeline holds standard collector configuration for a single operation, such as scraping MySQL metrics, built from components like prometheus.scrape, loki.source.file, or prometheus.remote_write.

You match a pipeline to collectors by their attributes. One pipeline can match many collectors, and one collector can match many pipelines. Each collector pulls the set of pipelines that match it, and that set is its remote configuration.

Each pipeline has a name, configuration content, and matching attributes.

Anatomy of a configuration pipeline

Pipeline components

ComponentDescription
NameUnique identifier for the pipeline
ContentStandard collector components for one operation
Matching attributesRules that assign the pipeline to collectors

Pipelines are reusable. Create once, assign to many collectors using attribute matching.

Script

In the previous slide, we talked about remote configuration as the solution to the scaling challenge. Now let’s get specific about what remote configuration actually is. In Fleet Management, remote configuration is delivered through something called a configuration pipeline.

A configuration pipeline is the unit of remote configuration in Fleet Management. When you “configure remotely,” you’re creating at least one pipeline. When collectors “pull configuration,” they’re pulling a set of one or more pipelines. The terms are closely related. Pipelines are how remote configuration works in practice.

Each pipeline has three parts: a unique name, the configuration content, and matching attributes that determine which collectors receive it.

The content is standard configuration: components like prometheus.scrape, loki.source.file, or prometheus.remote_write. You can write custom configs from scratch, or you can start from a Grafana Cloud integration that generates a preconfigured pipeline for common use cases.