Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Agent Flow mode Get started Collect OpenTelemetry data

Collect OpenTelemetry data

Grafana Agent Flow can be configured to collect OpenTelemetry-compatible data and forward it to any OpenTelemetry-compatible endpoint.

This topic describes how to:

• Configure OpenTelemetry data delivery
• Configure batching
• Receive OpenTelemetry data over OTLP

Components used in this topic

• otelcol.auth.basic
• otelcol.exporter.otlp
• otelcol.exporter.otlphttp
• otelcol.processor.batch
• otelcol.receiver.otlp

Before you begin

• Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
• Have a set of OpenTelemetry applications ready to push telemetry data to Grafana Agent Flow.
• Identify where Grafana Agent Flow will write received telemetry data.
• Be familiar with the concept of Components in Grafana Agent Flow.

Configure an OpenTelemetry Protocol exporter

Before components can receive OpenTelemetry data, you must have a component responsible for exporting the OpenTelemetry data. An OpenTelemetry exporter component is responsible for writing (that is, exporting) OpenTelemetry data to an external system.

In this task, we will use the otelcol.exporter.otlp component to send OpenTelemetry data to a server using the OpenTelemetry Protocol (OTLP). Once an exporter component is defined, other Grafana Agent Flow components can be used to forward data to it.

Refer to the list of available Components for the full list of otelcol.exporter components that can be used to export OpenTelemetry data.

To configure an otelcol.exporter.otlp component for exporting OpenTelemetry data using OTLP, complete the following steps:

1. Add the following otelcol.exporter.otlp component to your configuration file:

   river Copy

   otelcol.exporter.otlp "EXPORTER_LABEL" {
     client {
       url = "HOST:PORT"
     }
   }

   1. Replace EXPORTER_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.exporter.otlp components in the same configuration file.

   2. Replace HOST with the hostname or IP address of the server to send OTLP requests to.

   3. Replace PORT with the port of the server to send OTLP requests to.

2. If your server requires basic authentication, complete the following:

   1. Add the following otelcol.auth.basic component to your configuration file:

      river Copy

      otelcol.auth.basic "BASIC_AUTH_LABEL" {
        username = "USERNAME"
        password = "PASSWORD"
      }

      1. Replace BASIC_AUTH_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.auth.basic components in the same configuration file.

      2. Replace USERNAME with the basic authentication username to use.

      3. Replace PASSWORD with the basic authentication password or API key to use.

   2. Add the following line inside of the client block of your otelcol.exporter.otlp component:

      river Copy

      auth = otelcol.auth.basic.BASIC_AUTH_LABEL.handler

      1. Replace BASIC_AUTH_LABEL with the label used for the otelcol.auth.basic component in step 2.1.1.

3. If you have more than one server to export metrics to, create a new otelcol.exporter.otlp component for each additional server.

otelcol.exporter.otlp sends data using OTLP over gRPC (HTTP/2). To send to a server using HTTP/1.1, follow the steps above but use the otelcol.exporter.otlphttp component instead.

The following example demonstrates configuring otelcol.exporter.otlp with authentication and a component which forwards data to it:

river Copy

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.basic.credentials.handler
  }
}

otelcol.auth.basic "credentials" {
  // Retrieve credentials using environment variables.

  username = env("BASIC_AUTH_USER")
  password = env("API_KEY")
}

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

For more information on writing OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.exporter.otlp.

Configure batching

Production-ready Grafana Agent Flow configurations should not send OpenTelemetry data directly to an exporter for delivery. Instead, data is usually sent to one or more processor components that perform various transformations on the data.

Ensuring data is batched is a production-readiness step to improve the compression of data and reduce the number of outgoing network requests to external systems.

In this task, we will configure an otelcol.processor.batch component to batch data before sending it to our exporter.

Refer to the list of available Components for the full list of otelcol.processor components that can be used to process OpenTelemetry data. You can chain processors by having one processor send data to another processor.

To configure an otelcol.processor.batch component, complete the following steps:

1. Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.

2. Add the following otelcol.processor.batch component into your configuration file:

   river Copy

   otelcol.processor.batch "PROCESSOR_LABEL" {
     output {
       metrics = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
       logs    = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
       traces  = [otelcol.exporter.otlp.EXPORTER_LABEL.input]
     }
   }

   1. Replace PROCESSOR_LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.processor.batch components in the same configuration file.

   2. Replace EXPORTER_LABEL with the label for your existing otelcol.exporter.otlp component.

   3. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].

   4. To send batched data to another processor, replace the components in the output list with the processor components to use.

The following example demonstrates configuring a sequence of otelcol.processor components before ultimately being exported:

river Copy

otelcol.processor.memory_limiter "default" {
  check_interval = "1s"
  limit          = "1GiB"

  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on configuring OpenTelemetry data batching, refer to otelcol.processor.batch.

Configure an OpenTelemetry Protocol receiver

Grafana Agent Flow can be configured to receive OpenTelemetry metrics, logs, and traces. An OpenTelemetry receiver component is responsible for receiving OpenTelemetry data from an external system.

In this task, we will use the otelcol.receiver.otlp component to receive OpenTelemetry data over the network using the OpenTelemetry Protocol (OTLP). A receiver component can be configured to forward received data to other Grafana Agent Flow components.

Refer to the list of available Components for the full list of otelcol.receiver components that can be used to receive OpenTelemetry-compatible data.

To configure an otelcol.receiver.otlp component for receiving OTLP data, complete the following steps:

1. Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.

2. Optional: Follow Configure batching to improve compression and reduce the total amount of network requests.

3. Add the following otelcol.receiver.otlp component to your configuration file:

   river Copy

   otelcol.receiver.otlp "LABEL" {
     output {
       metrics = [COMPONENT_INPUT_LIST]
       logs    = [COMPONENT_INPUT_LIST]
       traces  = [COMPONENT_INPUT_LIST]
     }
   }

   1. Replace LABEL with a label to use for the component, such as default. The label chosen must be unique across all otelcol.receiver.otlp components in the same configuration file.

   2. Replace COMPONENT_INPUT_LIST with a comma-delimited list of component inputs to forward received data to. For example, to send data to an existing batch processor component, use otelcol.processor.batch.PROCESSOR_LABEL.input. To send data directly to an existing exporter component, use otelcol.exporter.otlp.EXPORTER_LABEL.input.

   3. To allow applications to send OTLP data over gRPC on port 4317, add the following to your otelcol.receiver.otlp component:

      river Copy

      grpc {
        endpoint = "HOST:4317"
      }

      1. Replace HOST with a host to listen to traffic on. It is recommended to use a narrowly-scoped listen address whenever possible. To listen on all network interfaces, replace HOST with 0.0.0.0.

   4. To allow applications to send OTLP data over HTTP/1.1 on port 4318, add the following to your otelcol.receiver.otlp component:

      river Copy

      http {
        endpoint = "HOST:4318"
      }

      1. Replace HOST with a host to listen to traffic on. It is recommended to use a narrowly-scoped listen address whenever possible. To listen on all network interfaces, replace HOST with 0.0.0.0.

   5. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].

The following example demonstrates configuring otelcol.receiver.otlp and sending it to an exporter:

river Copy

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on receiving OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.receiver.otlp.