Get started with Grafana Agent Flow on Grafana Labs

Collect and forward Prometheus metrics

Sun, 12 Apr 2026 09:27:09 +0000

Collect and forward Prometheus metrics

Grafana Agent Flow can be configured to collect Prometheus metrics and forward them to any Prometheus-compatible database.

This topic describes how to:

Configure metrics delivery
Collect metrics from Kubernetes Pods

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with Prometheus.
Have a set of Prometheus exports or applications exposing Prometheus metrics that you want to collect metrics from.
Identify where you will write collected metrics. Metrics may be written to Prometheus or Prometheus-compatible endpoints such as Grafana Mimir, Grafana Cloud, or Grafana Enterprise Metrics.
Be familiar with the concept of Components in Grafana Agent Flow.

Configure metrics delivery

Before components can collect Prometheus metrics, you must have a component responsible for writing those metrics somewhere.

The prometheus.remote_write component is responsible for delivering Prometheus metrics to one or Prometheus-compatible endpoints. Once a prometheus.remote_write component is defined, other Grafana Agent Flow components can be used to forward metrics to it.

To configure a prometheus.remote_write component for metrics delivery, complete the following steps:

Add the following prometheus.remote_write component to your configuration file:
Alloy
```
prometheus.remote_write "LABEL" {
  endpoint {
    url = "PROMETHEUS_URL"
  }
}
```
1. Replace LABEL with a label to use for the component, such as default. The label chosen must be unique across all prometheus.remote_write components in the same configuration file.
2. Replace PROMETHEUS_URL with the full URL of the Prometheus-compatible endpoint where metrics will be sent, such as https://prometheus-us-central1.grafana.net/api/prom/push.
If your endpoint requires basic authentication, paste the following inside of the endpoint block:
Alloy
```
basic_auth {
  username = "USERNAME"
  password = "PASSWORD"
}
```
1. Replace USERNAME with the basic authentication username to use.
2. Replace PASSWORD with the basic authentication password or API key to use.
If you have more than one endpoint to write metrics to, repeat the endpoint block for additional endpoints.

The following example demonstrates configuring prometheus.remote_write with multiple endpoints and mixed usage of basic authentication, and a prometheus.scrape component which forwards metrics to it:

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9090/api/prom/push"
  }

  endpoint {
    url = "https://prometheus-us-central1.grafana.net/api/prom/push"

    // Get basic authentication based on environment variables.
    basic_auth {
      username = env("REMOTE_WRITE_USERNAME")
      password = env("REMOTE_WRITE_PASSWORD")
    }
  }
}

prometheus.scrape "example" {
  // Collect metrics from the default listen address.
  targets = [{
    __address__ = "127.0.0.1:12345",
  }]

  forward_to = [prometheus.remote_write.default.receiver]
}

For more information on configuring metrics delivery, refer to prometheus.remote_write.

Collect metrics from Kubernetes Pods

Grafana Agent Flow can be configured to collect metrics from Kubernetes Pods by:

Discovering Kubernetes Pods to collect metrics from.
Collecting metrics from those discovered Pods.

To collect metrics from Kubernetes Pods, complete the following steps:

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Discover Kubernetes Pods:
1. Add the following discovery.kubernetes component to your configuration file to discover every Pod in the cluster across all Namespaces:
  Alloy
```
discovery.kubernetes "DISCOVERY_LABEL" {
  role = "pod"
}
```
  This will generate one Prometheus target for every exposed port on every discovered Pod.
  1. Replace DISCOVERY_LABEL with a label to use for the component, such as pods. The label chosen must be unique across all discovery.kubernetes components in the same configuration file.
2. To limit the Namespaces that Pods are discovered in, add the following block inside of the discovery.kubernetes component:
  Alloy
```
namespaces {
  own_namespace = true
  names         = [NAMESPACE_NAMES]
}
```
  1. If you don’t want to search for Pods in the Namespace Grafana Agent Flow is running in, set own_namespace to false.
  2. Replace NAMESPACE_NAMES with a comma-delimited list of strings representing Namespaces to search. Each string must be wrapped in double quotes. For example, "default","kube-system".
3. To use a field selector to limit the number of discovered Pods, add the following block inside of the discovery.kubernetes component:
  Alloy
```
selectors {
  role  = "pod"
  field = "FIELD_SELECTOR"
}
```
  1. Replace FIELD_SELECTOR with the Kubernetes field selector to use, such as metadata.name=my-service. For more information on field selectors, refer to the Kubernetes documentation on Field Selectors.
  2. Create additional selectors blocks for each field selector you want to apply.
4. To use a label selector to limit the number of discovered Pods, add the following block inside of the discovery.kubernetes component:
  Alloy
```
selectors {
  role  = "pod"
  label = "LABEL_SELECTOR"
}
```
  1. Replace LABEL_SELECTOR with the Kubernetes label selector to use, such as environment in (production, qa). For more information on label selectors, refer to the Kubernetes documentation on Labels and Selectors.
  2. Create additional selectors blocks for each label selector you want to apply.
Collect metrics from discovered Pods:
1. Add the following prometheus.scrape component to your configuration file:
  Alloy
```
prometheus.scrape "SCRAPE_LABEL" {
  targets    = discovery.kubernetes.DISCOVERY_LABEL.targets
  forward_to = [prometheus.remote_write.REMOTE_WRITE_LABEL.receiver]
}
```
  1. Replace SCRAPE_LABEL with a label to use for the component, such as pods. The label chosen must be unique across all prometeus.scrape components in the same configuration file.
  2. Replace DISCOVERY_LABEL with the label chosen for the discovery.kubernetes component in step 2.1.1.
  3. Replace REMOTE_WRITE_LABEL with the label chosen for your existing prometheus.remote_write component.

The following example demonstrates configuring Grafana Agent Flow to collect metrics from running production Kubernetes Pods in the default Namespace:

discovery.kubernetes "pods" {
  role = "pod"

  namespaces {
    own_namespace = false

    names = ["default"]
  }

  selectors {
    role  = "pod"
    label = "environment in (production)"
  }
}

prometheus.scrape "pods" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9090/api/prom/push"
  }
}

For more information on configuring Kubernetes service delivery and collecting metrics, refer to discovery.kubernetes and prometheus.scrape.

Collect metrics from Kubernetes Services

Grafana Agent Flow can be configured to collect metrics from Kubernetes Services by:

Discovering Kubernetes Services to collect metrics from.
Collecting metrics from those discovered Services.

To collect metrics from Kubernetes Services, complete the following steps:

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Discover Kubernetes Services:
1. Add the following discovery.kubernetes component to your configuration file to discover every Services in the cluster across all Namespaces:
  Alloy
```
discovery.kubernetes "DISCOVERY_LABEL" {
  role = "service"
}
```
  This will generate one Prometheus target for every exposed port on every discovered Service.
  1. Replace DISCOVERY_LABEL with a label to use for the component, such as services. The label chosen must be unique across all discovery.kubernetes components in the same configuration file.
2. To limit the Namespaces that Services are discovered in, add the following block inside of the discovery.kubernetes component:
  Alloy
```
namespaces {
  own_namespace = true
  names         = [NAMESPACE_NAMES]
}
```
  1. If you do not want to search for Services in the Namespace Grafana Agent Flow is running in, set own_namespace to false.
  2. Replace NAMESPACE_NAMES with a comma-delimited list of strings representing Namespaces to search. Each string must be wrapped in double quotes. For example, "default","kube-system".
3. To use a field selector to limit the number of discovered Services, add the following block inside of the discovery.kubernetes component:
  Alloy
```
selectors {
  role  = "service"
  field = "FIELD_SELECTOR"
}
```
  1. Replace FIELD_SELECTOR with the Kubernetes field selector to use, such as metadata.name=my-service. For more information on field selectors, refer to the Kubernetes documentation on Field Selectors.
  2. Create additional selectors blocks for each field selector you want to apply.
4. To use a label selector to limit the number of discovered Services, add the following block inside of the discovery.kubernetes component:
  Alloy
```
selectors {
  role  = "service"
  label = "LABEL_SELECTOR"
}
```
  1. Replace LABEL_SELECTOR with the Kubernetes label selector to use, such as environment in (production, qa). For more information on label selectors, refer to the Kubernetes documentation on Labels and Selectors.
  2. Create additional selectors blocks for each label selector you want to apply.
Collect metrics from discovered Services:
1. Add the following prometheus.scrape component to your configuration file:
  Alloy
```
prometheus.scrape "SCRAPE_LABEL" {
  targets    = discovery.kubernetes.DISCOVERY_LABEL.targets
  forward_to = [prometheus.remote_write.REMOTE_WRITE_LABEL.receiver]
}
```
  1. Replace SCRAPE_LABEL with a label to use for the component, such as services. The label chosen must be unique across all prometeus.scrape components in the same configuration file.
  2. Replace DISCOVERY_LABEL with the label chosen for the discovery.kubernetes component in step 2.1.1.
  3. Replace REMOTE_WRITE_LABEL with the label chosen for your existing prometheus.remote_write component.

The following example demonstrates configuring Grafana Agent Flow to collect metrics from running production Kubernetes Services in the default Namespace:

discovery.kubernetes "services" {
  role = "service"

  namespaces {
    own_namespace = false

    names = ["default"]
  }

  selectors {
    role  = "service"
    label = "environment in (production)"
  }
}

prometheus.scrape "services" {
  targets    = discovery.kubernetes.services.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9090/api/prom/push"
  }
}

For more information on configuring Kubernetes service delivery and collecting metrics, refer to discovery.kubernetes and prometheus.scrape.

Collect metrics from custom targets

Grafana Agent Flow can be configured to collect metrics from a custom set of targets without the need for service discovery.

To collect metrics from a custom set of targets, complete the following steps:

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Add the following prometheus.scrape component to your configuration file:
Alloy
```
prometheus.scrape "SCRAPE_LABEL" {
  targets    = [TARGET_LIST]
  forward_to = [prometheus.remote_write.REMOTE_WRITE_LABEL.receiver]
}
```
1. Replace SCRAPE_LABEL with a label to use for the component, such as custom_targets. The label chosen must be unique across all prometheus.scrape components in the same configuration file.
2. Replace TARGET_LIST with a comma-delimited list of Objects denoting the Prometheus target. Each object must conform to the following rules:
  - There must be an __address__ key denoting the HOST:PORT of the target to collect metrics from.
  - To explicitly specify which protocol to use, set the __scheme__ key to "http" or "https". If the __scheme__ key is not provided, the protocol to use is inherited by the settings of the prometheus.scrape component (default "http").
  - To explicitly specify which HTTP path to collect metrics from, set the __metrics_path__ key to the HTTP path to use. If the __metrics_path__ key is not provided, the path to use is inherited by the settings of the prometheus.scrape component (default "/metrics").
  - Add additional keys as desired to inject extra labels to collected metrics. Any label starting with two underscores (__) will be dropped prior to scraping.
3. Replace REMOTE_WRITE_LABEL with the label chosen for your existing prometheus.remote_write component.

The following example demonstrates configuring prometheus.scrape to collect metrics from a custom set of endpoints:

prometheus.scrape "custom_targets" {
  targets = [
    {
      __address__ = "prometheus:9090",
    },
    {
      __address__ = "mimir:8080",
      __scheme__  = "https",
    },
    {
      __address__      = "custom-application:80",
      __metrics_path__ = "/custom-metrics–path",
    },
    {
      __address__ = "grafana-agent:12345",
      application = "grafana-agent",
      environment = "production",
    },
  ]

  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9090/api/prom/push"
  }
}

Collect OpenTelemetry data

Sun, 12 Apr 2026 09:27:09 +0000

Collect OpenTelemetry data

Grafana Agent Flow can be configured to collect OpenTelemetry-compatible data and forward it to any OpenTelemetry-compatible endpoint.

This topic describes how to:

Configure OpenTelemetry data delivery
Configure batching
Receive OpenTelemetry data over OTLP

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
Have a set of OpenTelemetry applications ready to push telemetry data to Grafana Agent Flow.
Identify where Grafana Agent Flow will write received telemetry data.
Be familiar with the concept of Components in Grafana Agent Flow.

Configure an OpenTelemetry Protocol exporter

Before components can receive OpenTelemetry data, you must have a component responsible for exporting the OpenTelemetry data. An OpenTelemetry exporter component is responsible for writing (that is, exporting) OpenTelemetry data to an external system.

In this task, we will use the otelcol.exporter.otlp component to send OpenTelemetry data to a server using the OpenTelemetry Protocol (OTLP). Once an exporter component is defined, other Grafana Agent Flow components can be used to forward data to it.

Refer to the list of available Components for the full list of otelcol.exporter components that can be used to export OpenTelemetry data.

To configure an otelcol.exporter.otlp component for exporting OpenTelemetry data using OTLP, complete the following steps:

Add the following otelcol.exporter.otlp component to your configuration file:
Alloy
```
otelcol.exporter.otlp "EXPORTER_LABEL" {
  client {
    url = "HOST:PORT"
  }
}
```
1. Replace EXPORTER_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.exporter.otlp components in the same configuration file.
2. Replace HOST with the hostname or IP address of the server to send OTLP requests to.
3. Replace PORT with the port of the server to send OTLP requests to.
If your server requires basic authentication, complete the following:
1. Add the following otelcol.auth.basic component to your configuration file:
  Alloy
```
otelcol.auth.basic "BASIC_AUTH_LABEL" {
  username = "USERNAME"
  password = "PASSWORD"
}
```
  1. Replace BASIC_AUTH_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.auth.basic components in the same configuration file.
  2. Replace USERNAME with the basic authentication username to use.
  3. Replace PASSWORD with the basic authentication password or API key to use.
2. Add the following line inside of the client block of your otelcol.exporter.otlp component:
  Alloy
```
auth = otelcol.auth.basic.BASIC_AUTH_LABEL.handler
```
  1. Replace BASIC_AUTH_LABEL with the label used for the otelcol.auth.basic component in step 2.1.1.
If you have more than one server to export metrics to, create a new otelcol.exporter.otlp component for each additional server.

otelcol.exporter.otlp sends data using OTLP over gRPC (HTTP/2). To send to a server using HTTP/1.1, follow the steps above but use the otelcol.exporter.otlphttp component instead.

The following example demonstrates configuring otelcol.exporter.otlp with authentication and a component which forwards data to it:

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.basic.credentials.handler
  }
}

otelcol.auth.basic "credentials" {
  // Retrieve credentials using environment variables.

  username = env("BASIC_AUTH_USER")
  password = env("API_KEY")
}

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

For more information on writing OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.exporter.otlp.

Configure batching

Production-ready Grafana Agent Flow configurations should not send OpenTelemetry data directly to an exporter for delivery. Instead, data is usually sent to one or more processor components that perform various transformations on the data.

Ensuring data is batched is a production-readiness step to improve the compression of data and reduce the number of outgoing network requests to external systems.

In this task, we will configure an otelcol.processor.batch component to batch data before sending it to our exporter.

Refer to the list of available Components for the full list of otelcol.processor components that can be used to process OpenTelemetry data. You can chain processors by having one processor send data to another processor.

To configure an otelcol.processor.batch component, complete the following steps:

Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.
Add the following otelcol.processor.batch component into your configuration file:
Alloy
```
otelcol.processor.batch "PROCESSOR_LABEL" {
  output {
    metrics = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
    logs    = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
    traces  = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
  }
}
```
1. Replace PROCESSOR_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.processor.batch components in the same configuration file.
2. Replace EXPORTER_LABEL with the label for your existing otelcol.exporter.otlp component.
3. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].
4. To send batched data to another processor, replace the components in the output list with the processor components to use.

The following example demonstrates configuring a sequence of otelcol.processor components before ultimately being exported:

otelcol.processor.memory_limiter "default" {
  check_interval = "1s"
  limit          = "1GiB"

  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on configuring OpenTelemetry data batching, refer to otelcol.processor.batch.

Configure an OpenTelemetry Protocol receiver

Grafana Agent Flow can be configured to receive OpenTelemetry metrics, logs, and traces. An OpenTelemetry receiver component is responsible for receiving OpenTelemetry data from an external system.

In this task, we will use the otelcol.receiver.otlp component to receive OpenTelemetry data over the network using the OpenTelemetry Protocol (OTLP). A receiver component can be configured to forward received data to other Grafana Agent Flow components.

Refer to the list of available Components for the full list of otelcol.receiver components that can be used to receive OpenTelemetry-compatible data.

To configure an otelcol.receiver.otlp component for receiving OTLP data, complete the following steps:

Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.
Optional: Follow Configure batching to improve compression and reduce the total amount of network requests.
Add the following otelcol.receiver.otlp component to your configuration file:
Alloy
```
otelcol.receiver.otlp "LABEL" {
  output {
    metrics = [COMPONENT_INPUT_LIST]
    logs    = [COMPONENT_INPUT_LIST]
    traces  = [COMPONENT_INPUT_LIST]
  }
}
```
1. Replace LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.receiver.otlp components in the same configuration file.
2. Replace COMPONENT_INPUT_LIST with a comma-delimited list of component inputs to forward received data to. For example, to send data to an existing batch processor component, use otelcol.processor.batch.PROCESSOR_LABEL.input. To send data directly to an existing exporter component, use otelcol.exporter.otlp.EXPORTER_LABEL.input.
3. To allow applications to send OTLP data over gRPC on port 4317, add the following to your otelcol.receiver.otlp component:
  Alloy
```
grpc {
  endpoint = "HOST:4317"
}
```
  1. Replace HOST with a host to listen to traffic on. It is recommended to use a narrowly-scoped listen address whenever possible. To listen on all network interfaces, replace HOST with 0.0.0.0.
4. To allow applications to send OTLP data over HTTP/1.1 on port 4318, add the following to your otelcol.receiver.otlp component:
  Alloy
```
http {
  endpoint = "HOST:4318"
}
```
  1. Replace HOST with a host to listen to traffic on. It is recommended to use a narrowly-scoped listen address whenever possible. To listen on all network interfaces, replace HOST with 0.0.0.0.
5. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].

The following example demonstrates configuring otelcol.receiver.otlp and sending it to an exporter:

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on receiving OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.receiver.otlp.

Migrate from Prometheus to Grafana Agent Flow

Sun, 12 Apr 2026 09:27:09 +0000

Migrate from Prometheus to Grafana Agent Flow

The built-in Grafana Agent convert command can migrate your Prometheus configuration to a Grafana Agent Flow configuration.

This topic describes how to:

Convert a Prometheus configuration to a Grafana Agent Flow configuration.
Run a Prometheus configuration natively using Grafana Agent Flow.

Components used in this topic

Before you begin

You must have an existing Prometheus configuration.
You must have a set of Prometheus applications ready to push telemetry data to Grafana Agent Flow.
You must be familiar with the concept of Components in Grafana Agent Flow.

Convert a Prometheus configuration

To fully migrate your configuration fromPrometheus to Grafana Agent Flow, you must convert your Prometheus configuration into a Grafana Agent Flow configuration. This conversion will enable you to take full advantage of the many additional features available in Grafana Agent Flow.

In this task, we will use the convert CLI command to output a flow configuration from a Prometheus configuration.

Open a terminal window and run the following command:
static-binary
AGENT_MODE=flow grafana-agent convert --source-format=prometheus --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
flow-binary
grafana-agent-flow convert --source-format=prometheus --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
Replace the following:
- INPUT_CONFIG_PATH: The full path to the Prometheus configuration.
- OUTPUT_CONFIG_PATH: The full path to output the Grafana Agent Flow configuration.
Start Grafana Agent Flow using the new flow configuration from OUTPUT_CONFIG_PATH:

Debugging

If the convert command can’t convert a Prometheus configuration, diagnostic information is sent to stderr. You can bypass any non-critical issues and output the flow configuration using a best-effort conversion by including the --bypass-errors flag.

Caution

If you bypass the errors, the behavior of the converted configuration may not match the original Prometheus configuration.
Make sure you fully test the converted configuration before using it in a production environment.

AGENT_MODE=flow grafana-agent convert --source-format=prometheus --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=prometheus --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

You can also output a diagnostic report by including the --report flag.

AGENT_MODE=flow grafana-agent convert --source-format=prometheus --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=prometheus --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

Replace OUTPUT_REPORT_PATH with the output path for the report.

Using the example Prometheus configuration below, the diagnostic report provides the following information:

(Info) Converted scrape_configs job_name "prometheus" into...
  A prometheus.scrape.prometheus component
(Info) Converted 1 remote_write[s] "grafana-cloud" into...
  A prometheus.remote_write.default component

Run a Prometheus configuration

If you’re not ready to completely switch to a flow configuration, you can run Grafana Agent using your existing Prometheus configuration. The --config.format=prometheus flag tells Grafana Agent to convert your Prometheus configuration to a Grafana Agent Flow configuration and load it directly without saving the new configuration. This allows you to try Grafana Agent Flow without modifying your existing Prometheus configuration infrastructure.

In this task, we will use the run CLI command to run Grafana Agent Flow using a Prometheus configuration.

Start Grafana Agent Flow and include the command line flag --config.format=prometheus. Your configuration file must be a valid Prometheus configuration file rather than a Grafana Agent Flow configuration file.

Debugging

You can follow the convert CLI command debugging instructions to generate a diagnostic report.
Refer to the Grafana Agent Flow Debugging for more information about a running Grafana Agent Flow.
If your Prometheus configuration can’t be converted and loaded directly into Grafana Agent Flow, diagnostic information is sent to stderr. You can bypass any non-critical issues and start the Agent by including the --config.bypass-conversion-errors flag in addition to --config.format=prometheus.
Caution
```
If you bypass the errors, the behavior of the converted configuration may not match the original Prometheus configuration.
Do not use this flag in a production environment.
```

Example

This example demonstrates converting a Prometheus configuration file to a Grafana Agent Flow configuration file.

The following Prometheus configuration file provides the input for the conversion:

global:
  scrape_timeout:    45s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:12345"]

remote_write:
  - name: "grafana-cloud"
    url: "https://prometheus-us-central1.grafana.net/api/prom/push"
    basic_auth:
      username: USERNAME
      password: PASSWORD

The convert command takes the YAML file as input and outputs a River file.

AGENT_MODE=flow grafana-agent convert --source-format=prometheus --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=prometheus --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

The new flow configuration file looks like this:

prometheus.scrape "prometheus" {
  targets = [{
    __address__ = "localhost:12345",
  }]
  forward_to     = [prometheus.remote_write.default.receiver]
  job_name       = "prometheus"
  scrape_timeout = "45s"
}

prometheus.remote_write "default" {
  endpoint {
    name = "grafana-cloud"
    url  = "https://prometheus-us-central1.grafana.net/api/prom/push"

    basic_auth {
      username = "USERNAME"
      password = "PASSWORD"
    }

    queue_config {
      capacity             = 2500
      max_shards           = 200
      max_samples_per_send = 500
    }

    metadata_config {
      max_samples_per_send = 500
    }
  }
}

Limitations

Configuration conversion is done on a best-effort basis. Grafana Agent will issue warnings or errors where the conversion cannot be performed.

Once the configuration is converted, we recommend that you review the Grafana Agent Flow configuration file created and verify that it is correct before starting to use it in a production environment.

Furthermore, we recommend that you review the following checklist:

The following configurations aren’t available for conversion to Grafana Agent Flow: rule_files, alerting, remote_read, storage, and tracing. Any additional unsupported features are returned as errors during conversion.
Check if you are using any extra command line arguments with Prometheus that aren’t present in your configuration file. For example, --web.listen-address.
Metamonitoring metrics exposed by Grafana Agent Flow usually match Prometheus metamonitoring metrics but will use a different name. Make sure that you use the new metric names, for example, in your alerts and dashboards queries.
The logs produced by Grafana Agent Flow differ from those produced by Prometheus.
Grafana Agent exposes the Grafana Agent Flow UI.

Migrating from Grafana Agent Operator to Grafana Agent Flow

Sun, 12 Apr 2026 09:27:09 +0000

Migrating from Grafana Agent Operator to Grafana Agent Flow

With the release of Grafana Agent Flow, Grafana Agent Operator is no longer the recommended way to deploy Grafana Agent in Kubernetes. Some of the Operator functionality has been moved into Grafana Agent Flow itself, and the remaining functionality has been replaced by our Helm Chart.

The Monitor types (PodMonitor, ServiceMonitor, Probe, and LogsInstance) are all supported natively by Grafana Agent Flow. You are no longer required to use the Operator to consume those CRDs for dynamic monitoring in your cluster.
The parts of the Operator that deploy the Grafana Agent itself (GrafanaAgent, MetricsInstance, and LogsInstance CRDs) are deprecated. We now recommend operator users use the Grafana Agent Helm Chart to deploy Grafana Agent directly to your clusters.

This guide will provide some steps to get started with Grafana Agent Flow for users coming from Grafana Agent Operator.

Deploy Grafana Agent Flow with Helm

You will need to create a values.yaml file, which contains options for deploying your Grafana Agent. You may start with the default values and customize as you see fit, or start with this snippet, which should be a good starting point for what the Operator does:
YAML
```
agent:
  mode: 'flow'
  configMap:
    create: true
  clustering:
    enabled: true
controller:
  type: 'statefulset'
  replicas: 2
crds:
  create: false
```
This configuration will deploy Grafana Agent Flow as a StatefulSet using the built-in clustering functionality to allow distributing scrapes across all Grafana Agent Pods.

This is not the only deployment mode possible. For example, you may want to use a DaemonSet to collect host-level logs or metrics. See the Grafana Agent Flow deployment guide for more details about different topologies.
Create a Grafana Agent configuration file, agent.river.

We will be adding to this configuration in the next step as we convert MetricsInstances. You can add any additional configuration to this file as you desire.

Install the Grafana Helm repository:

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

Create a Helm release. You may name the release anything you like. Here we are installing a release named grafana-agent-metrics in the monitoring namespace.
shell
```
helm upgrade grafana-agent-metrics grafana/grafana-agent -i -n monitoring -f values.yaml --set-file agent.configMap.content=agent.river
```
This command uses the --set-file flag to pass the configuration file as a Helm value, so that we can continue to edit it as a regular River file.

Convert `MetricsIntances` to Grafana Agent Flow components

A MetricsInstance resource primarily defines:

The remote endpoint(s) Grafana Agent Flow should send metrics to.
Which PodMonitor, ServiceMonitor, and Probe resources this Grafana Agent should discover.

These functions can be done in Grafana Agent Flow with the prometheus.remote_write, prometheus.operator.podmonitors, prometheus.operator.servicemonitors, and prometheus.operator.probes components respectively.

This is a River sample that is equivalent to the MetricsInstance from our operator guide:


// read the credentials secret for remote_write authorization
remote.kubernetes.secret "credentials" {
  namespace = "monitoring"
  name = "primary-credentials-metrics"
}

prometheus.remote_write "primary" {
    endpoint {
        url = "https://PROMETHEUS_URL/api/v1/push"
        basic_auth {
            username = nonsensitive(remote.kubernetes.secret.credentials.data["username"])
            password = remote.kubernetes.secret.credentials.data["password"]
        }
    }
}

prometheus.operator.podmonitors "primary" {
    forward_to = [prometheus.remote_write.primary.receiver]
    // leave out selector to find all podmonitors in the entire cluster
    selector {
        match_labels = {instance = "primary"}
    }
}

prometheus.operator.servicemonitors "primary" {
    forward_to = [prometheus.remote_write.primary.receiver]
    // leave out selector to find all servicemonitors in the entire cluster
    selector {
        match_labels = {instance = "primary"}
    }
}

You will need to replace PROMETHEUS_URL with the actual endpoint you want to send metrics to.

This configuration will discover all PodMonitor, ServiceMonitor, and Probe resources in your cluster that match our label selector instance=primary. It will then scrape metrics from their targets and forward them to your remote write endpoint.

You may need to customize this configuration further if you use additional features in your MetricsInstance resources. Refer to the documentation for the relevant components for additional information:

Collecting Logs

Our current recommendation is to create an additional DaemonSet deployment of Grafana Agents to scrape logs.

We have components that can scrape pod logs directly from the Kubernetes API without needing a DaemonSet deployment. These are still considered experimental, but if you would like to try them, see the documentation for loki.source.kubernetes and loki.source.podlogs.

These values are close to what the Operator currently deploys for logs:

agent:
  mode: 'flow'
  configMap:
    create: true
  clustering:
    enabled: false
  controller:
    type: 'daemonset'
  mounts:
    # -- Mount /var/log from the host into the container for log collection.
    varlog: true

This command will install a release named grafana-agent-logs in the monitoring namespace:

helm upgrade grafana-agent-logs grafana/grafana-agent -i -n monitoring -f values-logs.yaml --set-file agent.configMap.content=agent-logs.river

This simple configuration will scrape logs for every pod on each node:

// read the credentials secret for remote_write authorization
remote.kubernetes.secret "credentials" {
  namespace = "monitoring"
  name      = "primary-credentials-logs"
}

discovery.kubernetes "pods" {
  role = "pod"
  // limit to pods on this node to reduce the amount we need to filter
  selectors {
    role  = "pod"
    field = "spec.nodeName=" + env("HOSTNAME")
  }
}

discovery.relabel "pod_logs" {
  targets = discovery.kubernetes.pods.targets
  rule {
    source_labels = ["__meta_kubernetes_namespace"]
    target_label  = "namespace"
  }
  rule {
    source_labels = ["__meta_kubernetes_pod_name"]
    target_label  = "pod"
  }
  rule {
    source_labels = ["__meta_kubernetes_pod_container_name"]
    target_label  = "container"
  }
  rule {
    source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_name"]
    separator     = "/"
    target_label  = "job"
  }
  rule {
    source_labels = ["__meta_kubernetes_pod_uid", "__meta_kubernetes_pod_container_name"]
    separator     = "/"
    action        = "replace"
    replacement   = "/var/log/pods/*$1/*.log"
    target_label  = "__path__"
  }
}

local.file_match "pod_logs" {
  path_targets = discovery.relabel.pod_logs.output
}

loki.source.file "pod_logs" {
  targets    = local.file_match.pod_logs.targets
  forward_to = [loki.process.pod_logs.receiver]
}

// basic processing to parse the container format. You can add additional processing stages
// to match your application logs.
loki.process "pod_logs" {
  stage.match {
    selector = "{tmp_container_runtime=\"containerd\"}"
    // the cri processing stage extracts the following k/v pairs: log, stream, time, flags
    stage.cri {}
    // Set the extract flags and stream values as labels
    stage.labels {
      values = {
        flags   = "",
        stream  = "",
      }
    }
  }

  // if the label tmp_container_runtime from above is docker parse using docker
  stage.match {
    selector = "{tmp_container_runtime=\"docker\"}"
    // the docker processing stage extracts the following k/v pairs: log, stream, time
    stage.docker {}

    // Set the extract stream value as a label
    stage.labels {
      values = {
        stream  = "",
      }
    }
  }

  // drop the temporary container runtime label as it is no longer needed
  stage.label_drop {
    values = ["tmp_container_runtime"]
  }

  forward_to = [loki.write.loki.receiver]
}

loki.write "loki" {
  endpoint {
    url = "https://LOKI_URL/loki/api/v1/push"
    basic_auth {
      username = nonsensitive(remote.kubernetes.secret.credentials.data["username"])
      password = remote.kubernetes.secret.credentials.data["password"]
    }
}
}

You will need to replace LOKI_URL with the actual endpoint of your Loki instance. The logging subsytem is very powerful and has many options for processing logs. For further details see the component documentation.

Integrations

The Integration CRD is not supported with Grafana Agent Flow, however all static mode integrations have an equivalent component in the prometheus.exporter namespace. The reference docs should help convert those integrations to their Flow equivalent.

Migrate from Promtail to Grafana Agent Flow

Sun, 12 Apr 2026 09:27:09 +0000

Migrate from Promtail to Grafana Agent Flow

The built-in Grafana Agent convert command can migrate your Promtail configuration to a Grafana Agent Flow configuration.

This topic describes how to:

Convert a Promtail configuration to a Grafana Agent Flow configuration.
Run a Promtail configuration natively using Grafana Agent Flow.

Components used in this topic

Before you begin

You must have an existing Promtail configuration.
You must be familiar with the concept of Components in Grafana Agent Flow.

Convert a Promtail configuration

To fully migrate from Promtail to Grafana Agent Flow, you must convert your Promtail configuration into a Grafana Agent Flow configuration. This conversion will enable you to take full advantage of the many additional features available in Grafana Agent Flow.

In this task, we will use the convert CLI command to output a flow configuration from a Promtail configuration.

Open a terminal window and run the following command:
static-binary
AGENT_MODE=flow grafana-agent convert --source-format=promtail --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
flow-binary
grafana-agent-flow convert --source-format=promtail --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
Replace the following:
- INPUT_CONFIG_PATH: The full path to the Promtail configuration.
- OUTPUT_CONFIG_PATH: The full path to output the flow configuration.
Start Grafana Agent Flow using the new flow configuration from OUTPUT_CONFIG_PATH:

Debugging

If the convert command cannot convert a Promtail configuration, diagnostic information is sent to stderr. You can bypass any non-critical issues and output the flow configuration using a best-effort conversion by including the --bypass-errors flag.

Caution
If you bypass the errors, the behavior of the converted configuration may not match the original Promtail configuration. Make sure you fully test the converted configuration before using it in a production environment.
static-binary
AGENT_MODE=flow grafana-agent convert --source-format=promtail --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
flow-binary
grafana-agent-flow convert --source-format=promtail --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

You can also output a diagnostic report by including the --report flag.

AGENT_MODE=flow grafana-agent convert --source-format=promtail --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=promtail --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

Replace OUTPUT_REPORT_PATH with the output path for the report.

Using the example Promtail configuration below, the diagnostic report provides the following information:

(Warning) If you have a tracing set up for Promtail, it cannot be migrated to Grafana Agent Flow automatically. Refer to the documentation on how to configure tracing in Grafana Agent Flow.
(Warning) The metrics from Grafana Agent Flow are different from the metrics emitted by Promtail. If you rely on Promtail's metrics, you must change your configuration, for example, your alerts and dashboards.

Run a Promtail configuration

If you’re not ready to completely switch to a flow configuration, you can run Grafana Agent using your existing Promtail configuration. The --config.format=promtail flag tells Grafana Agent to convert your Promtail configuration to Grafana Agent Flow and load it directly without saving the new configuration. This allows you to try Grafana Agent Flow without modifying your existing Promtail configuration infrastructure.

In this task, we will use the run CLI command to run Grafana Agent Flow using a Promtail configuration.

Start Grafana Agent Flow and include the command line flag --config.format=promtail. Your configuration file must be a valid Promtail configuration file rather than a Flow mode configuration file.

Debugging

You can follow the convert CLI command debugging instructions to generate a diagnostic report.
Refer to the Grafana Agent Flow Debugging for more information about running Grafana Agent Flow.
If your Promtail configuration can’t be converted and loaded directly into Grafana Agent, diagnostic information is sent to stderr. You can bypass any non-critical issues and start Grafana Agent by including the --config.bypass-conversion-errors flag in addition to --config.format=promtail.

Caution
If you bypass the errors, the behavior of the converted configuration may not match the original Promtail configuration. Do not use this flag in a production environment.

Example

This example demonstrates converting a Promtail configuration file to a Grafana Agent Flow configuration file.

The following Promtail configuration file provides the input for the conversion:

clients:
  - url: http://localhost/loki/api/v1/push
scrape_configs:
  - job_name: example
    static_configs:
      - targets:
          - localhost
        labels:
          __path__: /var/log/*.log

The convert command takes the YAML file as input and outputs a River file.

AGENT_MODE=flow grafana-agent convert --source-format=promtail --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=promtail --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

The new Grafana Agent Flow configuration file looks like this:

local.file_match "example" {
	path_targets = [{
		__address__ = "localhost",
		__path__    = "/var/log/*.log",
	}]
}

loki.source.file "example" {
	targets    = local.file_match.example.targets
	forward_to = [loki.write.default.receiver]
}

loki.write "default" {
	endpoint {
		url = "http://localhost/loki/api/v1/push"
	}
	external_labels = {}
}

Limitations

Configuration conversion is done on a best-effort basis. Grafana Agent will issue warnings or errors where the conversion can’t be performed.

Once the configuration is converted, we recommend that you review the Grafana Agent Flow configuration file created, and verify that it’s correct before starting to use it in a production environment.

Furthermore, we recommend that you review the following checklist:

Check if you are using any extra command line arguments with Promtail which aren’t present in your configuration file. For example, -max-line-size.
Check if you are setting any environment variables, whether expanded in the config file itself or consumed directly by Promtail, such as JAEGER_AGENT_HOST.
In Grafana Agent Flow, the positions file is saved at a different location. Refer to the loki.source.file documentation for more details. Check if you have any existing setup, for example, a Kubernetes Persistent Volume, that you must update to use the new positions file path.
Metamonitoring metrics exposed by the Flow Mode usually match Promtail metamonitoring metrics but will use a different name. Make sure that you use the new metric names, for example, in your alerts and dashboards queries.
Note that the logs produced by Grafana Agent Flow will differ from those produced by Promtail.
Note that Grafana Agent Flow exposes the Grafana Agent Flow UI, which differs from Promtail’s Web UI.

Migrate Grafana Agent Static to Grafana Agent Flow

Sun, 12 Apr 2026 09:27:09 +0000

Migrate from Grafana Agent Static to Grafana Agent Flow

The built-in Grafana Agent convert command can migrate your Static configuration to a Grafana Agent Flow configuration.

This topic describes how to:

Convert a Grafana Agent Static configuration to a Grafana Agent Flow configuration.
Run a Grafana Agent Static configuration natively using Grafana Agent Flow.

Components used in this topic

Before you begin

You must have an existing Grafana Agent Static configuration.
You must be familiar with the Components concept in Grafana Agent Flow.

Convert a Grafana Agent Static configuration

To fully migrate Grafana Agent Static to Grafana Agent Flow, you must convert your Static configuration into a Grafana Agent Flow configuration. This conversion will enable you to take full advantage of the many additional features available in Grafana Agent Flow.

In this task, we will use the convert CLI command to output a Grafana Agent Flow configuration from a Static configuration.

Open a terminal window and run the following command:
static-binary
AGENT_MODE=flow grafana-agent convert --source-format=static --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
flow-binary
grafana-agent-flow convert --source-format=static --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
Replace the following:
- INPUT_CONFIG_PATH: The full path to the Static configuration.
- OUTPUT_CONFIG_PATH: The full path to output the Grafana Agent Flow configuration.
Start Grafana Agent Flow using the new Grafana Agent Flow configuration from OUTPUT_CONFIG_PATH:

Debugging

If the convert command cannot convert a Static configuration, diagnostic information is sent to stderr. You can use the --bypass-errors flag to bypass any non-critical issues and output the Grafana Agent Flow configuration using a best-effort conversion.

Caution
If you bypass the errors, the behavior of the converted configuration may not match the original Static configuration. Make sure you fully test the converted configuration before using it in a production environment.
static-binary
AGENT_MODE=flow grafana-agent convert --source-format=static --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
flow-binary
grafana-agent-flow convert --source-format=static --bypass-errors --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

You can use the --report flag to output a diagnostic report.

AGENT_MODE=flow grafana-agent convert --source-format=static --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=static --report=OUTPUT_REPORT_PATH --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH
 ```

Replace OUTPUT_REPORT_PATH with the output path for the report.

Using the example Grafana Agent Static configuration below, the diagnostic report provides the following information:

(Warning) Please review your agent command line flags and ensure they are set in your Flow mode config file where necessary.

Run a Static mode configuration

If you’re not ready to completely switch to a Grafana Agent Flow configuration, you can run Grafana Agent using your existing Grafana Agent Static configuration. The --config.format=static flag tells Grafana Agent to convert your Static configuration to Grafana Agent Flow and load it directly without saving the new configuration. This allows you to try Grafana Agent Flow without modifying your existing Grafana Agent Static configuration infrastructure.

In this task, we will use the run CLI command to run Grafana Agent Flow using a Static configuration.

Start Grafana Agent Flow and include the command line flag --config.format=static. Your configuration file must be a valid Static configuration file.

Debugging

You can follow the convert CLI command debugging instructions to generate a diagnostic report.
Refer to the Grafana Agent Flow DebuggingUI for more information about running Grafana Agent Flow.
If your Static configuration can’t be converted and loaded directly into Grafana Agent Flow, diagnostic information is sent to stderr. You can use the --config.bypass-conversion-errors flag with --config.format=static to bypass any non-critical issues and start Grafana Agent Flow.

Caution
If you bypass the errors, the behavior of the converted configuration may not match the original Grafana Agent Static configuration. Do not use this flag in a production environment.

Example

This example demonstrates converting a Static configuration file to a Grafana Agent Flow configuration file.

The following Static configuration file provides the input for the conversion:

server:
  log_level: info

metrics:
  global:
    scrape_interval: 15s
    remote_write:
      - url: https://prometheus-us-central1.grafana.net/api/prom/push
        basic_auth:
          username: USERNAME
          password: PASSWORD
  configs:
    - name: test
      host_filter: false
      scrape_configs:
        - job_name: local-agent
          static_configs:
            - targets: ['127.0.0.1:12345']
              labels:
                cluster: 'localhost'

logs:
  global:
    file_watch_config:
      min_poll_frequency: 1s
      max_poll_frequency: 5s
  positions_directory: /var/lib/agent/data-agent
  configs:
    - name: varlogs
      scrape_configs:
        - job_name: varlogs
          static_configs:
            - targets:
              - localhost
              labels:
                job: varlogs
                host: mylocalhost
                __path__: /var/log/*.log
          pipeline_stages:
            - match:
                selector: '{filename="/var/log/*.log"}'
                stages:
                - drop:
                    expression: '^[^0-9]{4}'
                - regex:
                    expression: '^(?P<timestamp>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?P<level>[[:alpha:]]+)\] (?:\d+)\#(?:\d+): \*(?:\d+) (?P<message>.+)$'
                - pack:
                    labels:
                      - level
      clients:
        - url: https://USER_ID:API_KEY@logs-prod3.grafana.net/loki/api/v1/push

The convert command takes the YAML file as input and outputs a River file.

AGENT_MODE=flow grafana-agent convert --source-format=static --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

grafana-agent-flow convert --source-format=static --output=OUTPUT_CONFIG_PATH INPUT_CONFIG_PATH

The new Grafana Agent Flow configuration file looks like this:

prometheus.scrape "metrics_test_local_agent" {
	targets = [{
		__address__ = "127.0.0.1:12345",
		cluster     = "localhost",
	}]
	forward_to      = [prometheus.remote_write.metrics_test.receiver]
	job_name        = "local-agent"
	scrape_interval = "15s"
}

prometheus.remote_write "metrics_test" {
	endpoint {
		name = "test-3a2a1b"
		url  = "https://prometheus-us-central1.grafana.net/api/prom/push"

		basic_auth {
			username = "USERNAME"
			password = "PASSWORD"
		}

		queue_config { }

		metadata_config { }
	}
}

local.file_match "logs_varlogs_varlogs" {
	path_targets = [{
		__address__ = "localhost",
		__path__    = "/var/log/*.log",
		host        = "mylocalhost",
		job         = "varlogs",
	}]
}

loki.process "logs_varlogs_varlogs" {
	forward_to = [loki.write.logs_varlogs.receiver]

	stage.match {
		selector = "{filename=\"/var/log/*.log\"}"

		stage.drop {
			expression = "^[^0-9]{4}"
		}

		stage.regex {
			expression = "^(?P<timestamp>\\d{4}/\\d{2}/\\d{2} \\d{2}:\\d{2}:\\d{2}) \\[(?P<level>[[:alpha:]]+)\\] (?:\\d+)\\#(?:\\d+): \\*(?:\\d+) (?P<message>.+)$"
		}

		stage.pack {
			labels           = ["level"]
			ingest_timestamp = false
		}
	}
}

loki.source.file "logs_varlogs_varlogs" {
	targets    = local.file_match.logs_varlogs_varlogs.targets
	forward_to = [loki.process.logs_varlogs_varlogs.receiver]

	file_watch {
		min_poll_frequency = "1s"
		max_poll_frequency = "5s"
	}
}

loki.write "logs_varlogs" {
	endpoint {
		url = "https://USER_ID:API_KEY@logs-prod3.grafana.net/loki/api/v1/push"
	}
	external_labels = {}
}

Limitations

Configuration conversion is done on a best-effort basis. Grafana Agent will issue warnings or errors where the conversion cannot be performed.

Once the configuration is converted, we recommend that you review the Grafana Agent Flow configuration file, and verify that it is correct before starting to use it in a production environment.

Furthermore, we recommend that you review the following checklist:

The following configuration options aren’t available for conversion to Grafana Agent Flow: Integrations next, Traces, and Agent Management. Any additional unsupported features are returned as errors during conversion.
There is no gRPC server to configure for Grafana Agent Flow, as any non-default configuration will show as unsupported during the conversion.
Check if you are using any extra command line arguments with Static that aren’t present in your configuration file. For example, -server.http.address.
Check if you are using any environment variables in your Static configuration. These will be evaluated during conversion and you may want to replace them with the Grafana Agent Flow Standard library env function after conversion.
Review additional Prometheus Limitations for limitations specific to your Metrics configuration.
Review additional Promtail Limitations for limitations specific to your Logs configuration.
The logs produced by Grafana Agent Flow mode will differ from those produced by Static.
Grafana Agent exposes the Grafana Agent Flow UI.

OpenTelemetry to Grafana stack

Sun, 12 Apr 2026 09:27:09 +0000

OpenTelemetry to Grafana stack

You can configure Grafana Agent Flow to collect OpenTelemetry-compatible data and forward it to the Grafana stack

This topic describes how to:

Configure Grafana Agent Flow to send your data to Loki
Configure Grafana Agent Flow to send your data to Tempo
Configure Grafana Agent Flow to send your data to Mimir or Prometheus Remote Write

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
Have a set of OpenTelemetry applications ready to push telemetry data to Grafana Agent Flow.
Identify where Grafana Agent Flow will write received telemetry data.
Be familiar with the concept of Components in Grafana Agent Flow.
Complete the Collect open telemetry data getting started guide. You will pick up from where that guide ended.

The pipeline

You can start with the Grafana Agent Flow configuration you created in the previous getting started guide:

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

The pipeline currently looks like this:

Metrics, Logs, Traces: OTLP Receiver → batch processor → OTLP Exporter

You will implement the following pipelines to send your data to Loki, Tempo, and Mimir or Prometheus:

Metrics: OTel → batch processor → Mimir or Prometheus remote write
Logs: OTel → batch processor → Loki exporter
Traces: OTel → batch processor → OTel exporter

Grafana Loki

Grafana Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. Similar to Prometheus, to send from OTLP to Loki, we will do a passthrough from the otelcol.exporter.loki component to loki.write component.

otelcol.exporter.loki "default" {
	forward_to = [loki.write.default.receiver]
}

loki.write "default" {
	endpoint {
		url = "http://loki-endpoint:8080/loki/api/v1/push"
}

To use Loki with basic-auth, which is required with Grafana Cloud Loki, you must configure the loki.write component. You can get the Loki configuration from the Loki Details page in the Grafana Cloud Portal:

otelcol.exporter.loki "grafana_cloud_loki" {
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

loki.write "grafana_cloud_loki" {
	endpoint {
		url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"

		basic_auth {
			username = 5252
			password = env("GRAFANA_CLOUD_API_KEY")
		}
	}
}

Grafana Tempo

Grafana Tempo is an open source, easy-to-use, scalable distributed tracing backend. Tempo can ingest OTLP directly, and you can use the OTLP exporter to send the traces to Tempo.

otelcol.exporter.otlp "default" {
  client {
    endpoint = "tempo-server:4317"
  }
}

To use Tempo with basic-auth, which is required with Grafana Cloud Tempo, you must use the otelcol.auth.basic component. You can get the Tempo configuration from the Tempo Details page in the Grafana Cloud Portal:

otelcol.exporter.otlp "grafana_cloud_tempo" {
	client {
		endpoint = "tempo-us-central1.grafana.net:443"
		auth     = otelcol.auth.basic.grafana_cloud_tempo.handler
	}
}

otelcol.auth.basic "grafana_cloud_tempo" {
	username = 4094
	password = env("GRAFANA_CLOUD_API_KEY")
}

Grafana Mimir or Prometheus Remote Write

Prometheus Remote Write is a popular metrics transmission protocol supported by most metrics systems, including Grafana Mimir and Grafana Cloud. To send from OTLP to Prometheus, we do a passthrough from the otelcol.exporter.prometheus to the prometheus.remote_write component. The Prometheus remote write component in Grafana Agent Flow is a robust protocol implementation, including a Write Ahead Log (WAL) for resiliency.

otelcol.exporter.prometheus "default" {
	forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
	endpoint {
		url = "http://prometheus:9090/api/v1/write"
	}
}

To use Prometheus with basic-auth, which is required with Grafana Cloud Prometheus, you must configure the prometheus.remote_write component. You can get the Prometheus configuration from the Prometheus Details page in the Grafana Cloud Portal:

otelcol.exporter.prometheus "grafana_cloud_prometheus" {
        forward_to = [prometheus.remote_write.grafana_cloud_prometheus.receiver]
    }

prometheus.remote_write "grafana_cloud_prometheus" {
    endpoint {
        url = "https://prometheus-us-central1.grafana.net/api/prom/push"

        basic_auth {
            username = 12690
            password = env("GRAFANA_CLOUD_API_KEY")
        }
    }
}

Putting it all together

Instead of referencing otelcol.exporter.otlp.default.input in the output of otelcol.processor.batch, we need to reference the three exporters we set up. The final configuration becomes:

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.prometheus.grafana_cloud_prometheus.input]
    logs    = [otelcol.exporter.loki.grafana_cloud_loki.input]
    traces  = [otelcol.exporter.otlp.grafana_cloud_tempo.input]
  }
}

otelcol.exporter.otlp "grafana_cloud_tempo" {
	client {
		endpoint = "tempo-us-central1.grafana.net:443"
		auth     = otelcol.auth.basic.grafana_cloud_tempo.handler
	}
}

otelcol.auth.basic "grafana_cloud_tempo" {
	username = 4094
	password = env("GRAFANA_CLOUD_API_KEY")
}

otelcol.exporter.prometheus "grafana_cloud_prometheus" {
        forward_to = [prometheus.remote_write.grafana_cloud_prometheus.receiver]
    }

prometheus.remote_write "grafana_cloud_prometheus" {
    endpoint {
        url = "https://prometheus-us-central1.grafana.net/api/prom/push"

        basic_auth {
            username = 12690
            password = env("GRAFANA_CLOUD_API_KEY")
        }
    }
}

otelcol.exporter.loki "grafana_cloud_loki" {
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

loki.write "grafana_cloud_loki" {
	endpoint {
		url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"

		basic_auth {
			username = 5252
			password = env("GRAFANA_CLOUD_API_KEY")
		}
	}
}

Running Grafana Agent Flow now will give you the following:

AGENT_MODE=flow ./grafana-agent run agent-config.river
AGENT_MODE=flow ./grafana-agent run agent-config.river
ts=2023-05-09T09:37:15.300959Z level=info msg="running usage stats reporter"
ts=2023-05-09T09:37:15.300958Z level=info msg="now listening for http traffic" addr=127.0.0.1:12345
ts=2023-05-09T09:37:15.301104Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="starting complete graph evaluation"
ts=2023-05-09T09:37:15.301307Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=loki.write.grafana_cloud_loki duration=188.209µs
ts=2023-05-09T09:37:15.301334Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.loki.grafana_cloud_loki duration=18.791µs
ts=2023-05-09T09:37:15.303138Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="replaying WAL, this may take a while" dir=data-agent/prometheus.remote_write.grafana_cloud_prometheus/wal
ts=2023-05-09T09:37:15.303257Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="WAL segment loaded" segment=0 maxSegment=1
ts=2023-05-09T09:37:15.303302Z component=prometheus.remote_write.grafana_cloud_prometheus level=info subcomponent=wal msg="WAL segment loaded" segment=1 maxSegment=1
ts=2023-05-09T09:37:15.303507Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting WAL watcher" queue=7f623a
ts=2023-05-09T09:37:15.303515Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting scraped metadata watcher"
ts=2023-05-09T09:37:15.303522Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=prometheus.remote_write.grafana_cloud_prometheus duration=2.181958ms
ts=2023-05-09T09:37:15.303557Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.prometheus.grafana_cloud_prometheus duration=30.083µs
ts=2023-05-09T09:37:15.303611Z component=prometheus.remote_write.grafana_cloud_prometheus subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Replaying WAL" queue=7f623a
ts=2023-05-09T09:37:15.303618Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.auth.basic.grafana_cloud_tempo duration=52.5µs
ts=2023-05-09T09:37:15.303694Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.otlp.grafana_cloud_tempo duration=70.375µs
ts=2023-05-09T09:37:15.303782Z component=otelcol.processor.memory_limiter.default level=info msg="Memory limiter configured" limit_mib=150 spike_limit_mib=30 check_interval=1s
ts=2023-05-09T09:37:15.303802Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.memory_limiter.default duration=100.334µs
ts=2023-05-09T09:37:15.303853Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.batch.default duration=44.75µs
ts=2023-05-09T09:37:15.303948Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.receiver.otlp.default duration=87.333µs
ts=2023-05-09T09:37:15.303968Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=tracing duration=10.792µs
ts=2023-05-09T09:37:15.303981Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=logging duration=9µs
ts=2023-05-09T09:37:15.303987Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished complete graph evaluation" duration=2.960333ms
ts=2023-05-09T09:37:15.304Z level=info msg="scheduling loaded components"
ts=2023-05-09T09:37:15.304109Z component=otelcol.receiver.otlp.default level=info msg="Starting GRPC server" endpoint=0.0.0.0:4317
ts=2023-05-09T09:37:15.304234Z component=otelcol.receiver.otlp.default level=info msg="Starting HTTP server" endpoint=0.0.0.0:4318

You can now check the pipeline graphically by visiting http://localhost:12345/graph

Configure Grafana Agent clustering in an existing installation

Sun, 12 Apr 2026 09:27:09 +0000

Configure Grafana Agent Flow clustering in an existing installation

You can configure Grafana Agent Flow to run with clustering so that individual Grafana Agents can work together for workload distribution and high availability.

Note: Clustering is a beta feature. Beta features are subject to breaking changes and may be replaced with equivalent functionality that covers the same use case.

This topic describes how to add clustering to an existing installation.

Configure Grafana Agent Flow clustering with Helm Chart

This section guides you through enabling clustering when Grafana Agent Flow is installed on Kubernetes using the Grafana Agent Helm chart.

Before you begin

Ensure that your values.yaml file has controller.type set to statefulset.

Steps

To configure clustering:

Amend your existing values.yaml file to add clustering.enabled=true inside of the agent block:
YAML
```
agent:
  clustering:
    enabled: true
```
Upgrade your installation to use the new values.yaml file:
Bash
```
helm upgrade RELEASE_NAME -f values.yaml
```
Replace RELEASE_NAME with the name of the installation you chose when you installed the Helm chart.
Use the Grafana Agent Flow UI to verify the cluster status:
1. Click Clustering in the navigation bar.
2. Ensure that all expected nodes appear in the resulting table.

Distribute Prometheus metrics scrape load

Sun, 12 Apr 2026 09:27:09 +0000

Distribute Prometheus metrics scrape load

A good predictor for the size of an Grafana Agent Flow deployment is the number of Prometheus targets each Grafana Agent scrapes. Clustering with target auto-distribution allows a fleet of Grafana Agents to work together to dynamically distribute their scrape load, providing high-availability.

Note: Clustering is a beta feature. Beta features are subject to breaking changes and may be replaced with equivalent functionality that covers the same use case.

Before you begin

Familiarize yourself with how to configure existing Grafana Agent Flow installations.
Configure Prometheus metrics collection.
Configure clustering.
Ensure that all of your clustered Grafana Agents have the same configuration file.

Steps

To distribute Prometheus metrics scrape load with clustering:

Add the following block to all prometheus.scrape components which should use auto-distribution:
Alloy
```
clustering {
  enabled = true
}
```
Restart or reload Grafana Agents for them to use the new configuration.
Validate that auto-distribution is functioning:
1. Using the Grafana Agent UI on each Grafana Agent, navigate to the details page for one of the prometheus.scrape components you modified.
2. Compare the Debug Info sections between two different Grafana Agent to ensure that they’re not scraping the same sets of targets.

Get started with Grafana Agent Flow on Grafana Labs

Collect and forward Prometheus metrics

Collect and forward Prometheus metrics

Components used in this topic

Before you begin

Configure metrics delivery

Collect metrics from Kubernetes Pods

Collect metrics from Kubernetes Services

Collect metrics from custom targets

Collect OpenTelemetry data

Collect OpenTelemetry data

Components used in this topic

Before you begin

Configure an OpenTelemetry Protocol exporter

Configure batching

Configure an OpenTelemetry Protocol receiver

Migrate from Prometheus to Grafana Agent Flow

Migrate from Prometheus to Grafana Agent Flow

Components used in this topic

Before you begin

Convert a Prometheus configuration

Debugging

Run a Prometheus configuration

Debugging

Example

Limitations

Migrating from Grafana Agent Operator to Grafana Agent Flow

Migrating from Grafana Agent Operator to Grafana Agent Flow

Deploy Grafana Agent Flow with Helm

Convert MetricsIntances to Grafana Agent Flow components

Collecting Logs

Integrations

Migrate from Promtail to Grafana Agent Flow

Migrate from Promtail to Grafana Agent Flow

Components used in this topic

Before you begin

Convert a Promtail configuration

Debugging

Run a Promtail configuration

Debugging

Example

Limitations

Migrate Grafana Agent Static to Grafana Agent Flow

Migrate from Grafana Agent Static to Grafana Agent Flow

Components used in this topic

Before you begin

Convert a Grafana Agent Static configuration

Debugging

Run a Static mode configuration

Debugging

Example

Limitations

OpenTelemetry to Grafana stack

OpenTelemetry to Grafana stack

Components used in this topic

Before you begin

The pipeline

Grafana Loki

Grafana Tempo

Grafana Mimir or Prometheus Remote Write

Putting it all together

Configure Grafana Agent clustering in an existing installation

Configure Grafana Agent Flow clustering in an existing installation

Configure Grafana Agent Flow clustering with Helm Chart

Before you begin

Steps

Distribute Prometheus metrics scrape load

Distribute Prometheus metrics scrape load

Before you begin

Steps

Convert `MetricsIntances` to Grafana Agent Flow components