Collect and forward data with Grafana Alloy on Grafana Labs

Choose a Grafana Alloy component

Mon, 30 Mar 2026 15:47:22 +0000

Choose a Grafana Alloy component

Components are the building blocks of Grafana Alloy, and there is a large number of them. The components you select and configure depend on the telemetry signals you want to collect.

Metrics for infrastructure

Use prometheus.* components to collect infrastructure metrics. This gives you the best experience with Grafana Infrastructure Observability.

For example, you can get metrics for a Linux host using prometheus.exporter.unix, and metrics for a MongoDB instance using prometheus.exporter.mongodb.

You can also scrape any Prometheus endpoint using prometheus.scrape. Use discovery.* components to find targets for prometheus.scrape.

Metrics for applications

Use otelcol.receiver.* components to collect application metrics. This gives you the best experience with Grafana Application Observability, which is OpenTelemetry-native.

For example, use otelcol.receiver.otlp to collect metrics from OpenTelemetry-instrumented applications.

If your application is already instrumented with Prometheus metrics, there is no need to use otelcol.* components. Use prometheus.* components for the entire pipeline and send the metrics using prometheus.remote_write.

Logs from infrastructure

Use loki.* components to collect infrastructure logs. The loki.* components label your logs in a way that resembles Prometheus metrics. This makes it easy to correlate infrastructure metrics collected by prometheus.* components with logs collected by loki.* components.

For example, the label that both prometheus.* and loki.* components would use for a Kubernetes namespace is called namespace. On the other hand, gathering logs using an otelcol.* component might use the OpenTelemetry semantics label called k8s.namespace.name, which wouldn’t correspond to the namespace label that’s common in the Prometheus ecosystem.

Logs from applications

Use otelcol.receiver.* components to collect application logs. This gathers the application logs in an OpenTelemetry-native way, making it easier to correlate the logs with OpenTelemetry metrics and traces coming from the application. All application telemetry must follow the OpenTelemetry semantic conventions, simplifying this correlation.

For example, if your application runs on Kubernetes, every trace, log, and metric can have a k8s.namespace.name resource attribute.

Traces

Use otelcol.receiver.* components to collect traces.

If your application isn’t yet instrumented for tracing, use beyla.ebpf to generate traces for it automatically.

Profiles

Use pyroscope.* components to collect profiles.

Frontend telemetry

In order to use Grafana Cloud Frontend Observability, you have to collect and forward frontend telemetry using otelcol.receiver.faro and otelcol.exporter.faro.

You can also gather frontend telemetry using faro.receiver and send it to Grafana Cloud, but Grafana Cloud Frontend Observability will not work and you will need to create your own dashboards.

faro.receiver is recommended only for a self-hosted Grafana setup.

Set up meta-monitoring to collect Alloy telemetry

Mon, 30 Mar 2026 15:47:22 +0000

Set up meta-monitoring to collect Alloy telemetry

You can configure Alloy to collect its own telemetry and forward it to the backend of your choosing.

This topic describes how to collect and forward metrics, logs, and traces data from Alloy.

Components and configuration blocks used in this topic

Before you begin

Identify where to send Alloy’s telemetry data.
Be familiar with the concept of Components in Alloy.

Meta-monitoring metrics

Alloy exposes its internal metrics using the Prometheus exposition format.

The metrics are exposed on a /metrics endpoint on an address configurable via the --server.http.listen-addr command line flag. For example, 127.0.0.1:12345/metrics. However, it’s also possible to access the same metrics using the prometheus.exporter.self component.

In this task, you use the prometheus.exporter.self and prometheus.scrape components to scrape Alloy’s internal metrics and forward it to compatible Alloy components.

Add the following prometheus.exporter.self component to your configuration. The component accepts no arguments.
Alloy
```
prometheus.exporter.self "<SELF_LABEL>" {
}
```
Replace the following:
- <SELF_LABEL>: The label for the component such as default or metamonitoring. The label must be unique across all prometheus.exporter.self components in the same configuration file.
Add the following prometheus.scrape component to your configuration file.
Alloy
```
prometheus.scrape "<SCRAPE_LABEL>" {
  targets    = prometheus.exporter.self.<SELF_LABEL>.targets
  forward_to = [<METRICS_RECEIVER_LIST>]
}
```
Replace the following:
- <SCRAPE_LABEL>: The label for the scrape component such as default. The label must be unique across all prometheus.scrape components in the same configuration file.
- <METRICS_RECEIVER_LIST>: A comma-delimited list of component receivers to forward metrics to. For example, to send to a remote write component, use prometheus.remote_write.WRITE_LABEL.receiver. Similarly, to send data to a relabeling component, use prometheus.relabel.PROCESS_LABEL.receiver. To use data in the OTLP format, you can send data to a converter component, like otelcol.receiver.prometheus.OTEL.receiver.

The following example demonstrates configuring a possible sequence of components.

prometheus.exporter.self "default" {
}

prometheus.scrape "metamonitoring" {
  targets    = prometheus.exporter.self.default.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Meta-monitoring logs

The logging block defines the logging behavior of Alloy.

In this task, you use the logging block to forward Alloy’s logs to a compatible component. The block is specified without a label and can only be provided once per configuration file.

Add the following logging configuration block to the top level of your configuration file.
Alloy
```
logging {
  level    = "<LOG_LEVEL>"
  format   = "<LOG_FORMAT>"
  write_to = [<LOGS_RECEIVER_LIST>]
}
```
Replace the following:
- <LOG_LEVEL>: The log level to use for Alloy’s logs. If the attribute isn’t set, it defaults to info.
- <LOG_FORMAT>: The log format to use for Alloy’s logs. If the attribute isn’t set, it defaults to logfmt.
- <LOGS_RECEIVER_LIST>: A comma-delimited list of component receivers to forward logs to. For example, to send to a processing component, use loki.process.PROCESS_LABEL.receiver. Similarly, to send data to a relabeling component, use loki.relabel.PROCESS_LABEL.receiver. To use data in the OTLP format, you can send data to a converter component, like otelcol.receiver.loki.OTEL.receiver.

The following example demonstrates configuring the logging block and sending to a compatible component.

logging {
  level    = "warn"
  format   = "json"
  write_to = [loki.write.default.receiver]
}

loki.write "default" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
  }
}

Meta-monitoring traces

The tracing block defines the tracing behavior of Alloy.

In this task, you use the tracing block to forward Alloy internal traces to a compatible component. The block is specified without a label and can only be provided once per configuration file.

Add the following tracing configuration block to the top level of your configuration file.
Alloy
```
tracing {
  sampling_fraction = <SAMPLING_FRACTION>
  write_to          = [<TRACES_RECEIVER_LIST>]
}
```
Replace the following:
- <SAMPLING_FRACTION>: The fraction of traces to keep. If the attribute isn’t set, it defaults to 0.1.
- <TRACES_RECEIVER_LIST>: A comma-delimited list of component receivers to forward traces to. For example, to send to an OpenTelemetry exporter component use otelcol.exporter.otlp.EXPORT_LABEL.input.

The following example demonstrates configuring the tracing block and sending to a compatible component.

tracing {
  sampling_fraction = 0.1
  write_to          = [otelcol.exporter.otlp.default.input]
}

otelcol.exporter.otlp "default" {
    client {
        endpoint = "tempo:4317"
    }
}

Receive traces and metrics from Datadog-instrumented applications

Mon, 30 Mar 2026 15:47:22 +0000

Receive traces and metrics from Datadog-instrumented applications

You can configure Alloy to collect Datadog traces and metrics and forward them to any OpenTelemetry-compatible database.

This topic describes how to:

Configure Alloy to send traces and metrics.
Configure the Alloy Datadog Receiver.
Configure the Datadog Agent to forward traces and metrics to the Alloy Datadog Receiver.

Before you begin

Ensure that at least one instance of the Datadog Agent is collecting metrics and traces.
Identify where to write the collected telemetry. Metrics can be written to Prometheus or any other OpenTelemetry-compatible database such as Grafana Mimir, Grafana Cloud, or Grafana Enterprise Metrics. Traces can be written to Grafana Tempo, Grafana Cloud, or Grafana Enterprise Traces.
Be familiar with the concept of Components in Alloy.

Configure Alloy to send traces and metrics

Before components can collect Datadog telemetry signals, you must have a component responsible for writing this telemetry somewhere.

The otelcol.exporter.otlp component is responsible for delivering OTLP data to OpenTelemetry-compatible endpoints.

Add the following otelcol.exporter.otlp component to your configuration file.
Alloy
```
otelcol.exporter.otlp "default" {
  client {
    endpoint = "<OTLP_ENDPOINT_URL>"
    auth     = otelcol.auth.basic.auth.handler
  }
}
```
Replace the following:
- <OTLP_ENDPOINT_URL>: The full URL of the OpenTelemetry-compatible endpoint where metrics and traces are sent, such as https://otlp-gateway-prod-eu-west-2.grafana.net/otlp.
If your endpoint requires basic authentication, paste the following inside the endpoint block.
Alloy
```
basic_auth {
  username = "<USERNAME>"
  password = "<PASSWORD>"
}
```
Replace the following:
- <USERNAME>: The basic authentication username.
- <PASSWORD>: The basic authentication password or API key.

Configure the Alloy Datadog Receiver

Add the following otelcol.processor.batch component to your configuration file.

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Add the following otelcol.processor.deltatocumulative component to your configuration file.
Alloy
```
otelcol.processor.deltatocumulative "default" {
  max_stale = "<MAX_STALE>"
  max_streams = <MAX_STREAMS>
  output {
    metrics = [otelcol.processor.batch.default.input]
  }
}
```
Replace the following:
- <MAX_STALE>: How long until a series not receiving new samples is removed, such as “5m”.
- <MAX_STREAMS>: The upper limit of streams to track. New streams exceeding this limit are dropped.
Add the following otelcol.receiver.datadog component to your configuration file.
Alloy
```
otelcol.receiver.datadog "default" {
  endpoint = "<HOST>:<PORT>"
  output {
    metrics = [otelcol.processor.deltatocumulative.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}
```
Replace the following:
- <HOST>: The host address where the receiver listens.
- <PORT>: The port where the receiver listens.
If your endpoint requires basic authentication, paste the following inside the endpoint block.
Alloy
```
basic_auth {
  username = "<USERNAME>"
  password = "<PASSWORD>"
}
```
Replace the following:
- <USERNAME>: The basic authentication username.
- <PASSWORD>: The basic authentication password or API key.

Configure Datadog Agent to forward telemetry to the Alloy Datadog Receiver

You can set up your Datadog Agent to forward Datadog metrics and traces simultaneously to Alloy and Datadog.

We recommend this approach for current Datadog users who want to try using Alloy.

Add the following environment variables to your Datadog-agent installation.
Bash
```
DD_ADDITIONAL_ENDPOINTS='{"http://<DATADOG_RECEIVER_HOST>:<DATADOG_RECEIVER_PORT>": ["datadog-receiver"]}'
DD_APM_ADDITIONAL_ENDPOINTS='{"http://<DATADOG_RECEIVER_HOST>:<DATADOG_RECEIVER_PORT>": ["datadog-receiver"]}'
```
DD_ADDITIONAL_ENDPOINTS is used for forwarding metrics, whereas DD_APM_ADDITIONAL_ENDPOINTS is for traces.

Replace the following:
- <DATADOG_RECEIVER_HOST>: The hostname where the Alloy receiver is found.
- <DATADOG_RECEIVER_PORT>: The port where the Alloy receiver is exposed.

Alternatively, you might want your Datadog Agent to send metrics only to Alloy. You can do this by setting up your Datadog Agent in the following way:

Replace the DD_URL in the configuration YAML:

 dd_url: http://<DATADOG_RECEIVER_HOST>:<DATADOG_RECEIVER_PORT>

Or by setting an environment variable:

DD_URL='{"http://<DATADOG_RECEIVER_HOST>:<DATADOG_RECEIVER_PORT>": ["datadog-receiver"]}'

Run Alloy

The otelcol.receiver.datadog component is experimental. To use this component, you need to start Alloy with additional command line flags:

alloy run config.alloy --stability.level=experimental

Collect Kubernetes logs and forward them to Loki

Mon, 30 Mar 2026 15:47:22 +0000

Collect Kubernetes logs and forward them to Loki

You can configure Alloy to collect logs and forward them to a Loki database.

To collect Kubernetes logs, you:

Configure a loki.write component to deliver logs.
Set up collection components for system logs, Pod logs, or Kubernetes Cluster Events.

Components used

Before you begin

Ensure that you’re familiar with log labeling when working with Loki.
Identify where you want to write collected logs. You can write logs to Loki endpoints such as Grafana Loki, Grafana Cloud, or Grafana Enterprise Logs.
Ensure that you’re familiar with the concept of Components in Alloy.

Configure logs delivery

Before components can collect logs, you must have a component responsible for writing those logs somewhere.

The loki.write component delivers logs to a Loki endpoint. After you define a loki.write component, other Alloy components can forward logs to it.

To configure a loki.write component for logs delivery, complete the following steps:

Add the following loki.write component to your configuration file.
Alloy
```
loki.write "<LABEL>" {
  endpoint {
    url = "<LOKI_URL>"
  }
}
```
Replace the following:
- <LABEL>: The label for the component, such as default. The label you use must be unique across all loki.write components in the same configuration file.
- <LOKI_URL>: The full URL of the Loki endpoint where you send logs, such as https://logs-us-central1.grafana.net/loki/api/v1/push.
If your endpoint requires basic authentication, paste the following inside the endpoint block.
Alloy
```
basic_auth {
  username = "<USERNAME>"
  password = "<PASSWORD>"
}
```
Replace the following:
- <USERNAME>: The basic authentication username.
- <PASSWORD>: The basic authentication password or API key.
If you have more than one endpoint to write logs to, repeat the endpoint block for additional endpoints.

The following example demonstrates configuring loki.write with multiple endpoints, mixed usage of basic authentication, and a loki.source.file component. The loki.source.file component collects logs from the file system on the Alloy container.

loki.write "default" {
  endpoint {
    url = "http://localhost:3100/loki/api/v1/push"
  }

  endpoint {
    url = "https://logs-us-central1.grafana.net/loki/api/v1/push"

    // Get basic authentication based on environment variables.
    basic_auth {
      username = "<USERNAME>"
      password = "<PASSWORD>"
    }
  }
}

loki.source.file "example" {
  // Collect logs from the default listen address.
  targets = [
    {__path__ = "/tmp/foo.txt", "color" = "pink"},
    {__path__ = "/tmp/bar.txt", "color" = "blue"},
    {__path__ = "/tmp/baz.txt", "color" = "grey"},
  ]

  forward_to = [loki.write.default.receiver]
}

Replace the following:

<USERNAME>: The remote write username.
<PASSWORD>: The remote write password.

For more information on configuring logs delivery, refer to loki.write.

Collect logs from Kubernetes

You can configure Alloy to collect all kinds of logs from Kubernetes:

System logs
Pod logs
Kubernetes Events

With the component architecture, you can follow one or all of the following sections. Complete the Configure logs delivery section first, then proceed to the log type you want to collect.

System logs

To collect system logs, use the following components:

local.file_match: Discovers files on the local file system.
loki.source.file: Reads log entries from files.
loki.write: Sends logs to the Loki endpoint. Configure this component in the Configure logs delivery section.

The following example collects system logs:

// local.file_match discovers files on the local filesystem using glob patterns and the doublestar library. It returns an array of file paths.
local.file_match "node_logs" {
  path_targets = [{
      // Monitor syslog to scrape node-logs
      __path__  = "/var/log/syslog",
      job       = "node/syslog",
      node_name = sys.env("HOSTNAME"),
      cluster   = <CLUSTER_NAME>,
  }]
}

// loki.source.file reads log entries from files and forwards them to other loki.* components.
// You can specify multiple loki.source.file components by giving them different labels.
loki.source.file "node_logs" {
  targets    = local.file_match.node_logs.targets
  forward_to = [loki.write.<WRITE_COMPONENT_NAME>.receiver]
}

Replace the following values:

<CLUSTER_NAME>: The label for this specific Kubernetes cluster, such as production or us-east-1.
<WRITE_COMPONENT_NAME>: The name of your loki.write component, such as default.

Pod logs

Tip
You can also get Pod logs through the log files on each node, but that approach requires system privileges.

Note
When you deploy Alloy as a DaemonSet, ensure that you configure discovery to only collect logs from the same node.

Use the following components to collect Pod logs:

discovery.kubernetes: Discovers Pod information and lists them for components to use.
discovery.relabel: Enforces relabeling strategies on the list of Pods.
loki.source.kubernetes: Tails logs from a list of Kubernetes Pods targets.
loki.process: Modifies logs before sending them to the next component.
loki.write: Sends logs to the Loki endpoint. Configure this component in the Configure logs delivery section.

The following example uses the Kubernetes API to collect logs, which doesn’t require system privileges:

// discovery.kubernetes allows you to find scrape targets from Kubernetes resources.
// It watches cluster state and ensures targets are continually synced with what is currently running in your cluster.
discovery.kubernetes "pod" {
  role = "pod"
  // Restrict to pods on the node to reduce cpu & memory usage
  selectors {
    role = "pod"
    field = "spec.nodeName=" + coalesce(sys.env("HOSTNAME"), constants.hostname)
  }
}

// discovery.relabel rewrites the label set of the input targets by applying one or more relabeling rules.
// If no rules are defined, then the input targets are exported as-is.
discovery.relabel "pod_logs" {
  targets = discovery.kubernetes.pod.targets

  // Label creation - "namespace" field from "__meta_kubernetes_namespace"
  rule {
    source_labels = ["__meta_kubernetes_namespace"]
    action = "replace"
    target_label = "namespace"
  }

  // Label creation - "pod" field from "__meta_kubernetes_pod_name"
  rule {
    source_labels = ["__meta_kubernetes_pod_name"]
    action = "replace"
    target_label = "pod"
  }

  // Label creation - "container" field from "__meta_kubernetes_pod_container_name"
  rule {
    source_labels = ["__meta_kubernetes_pod_container_name"]
    action = "replace"
    target_label = "container"
  }

  // Label creation -  "app" field from "__meta_kubernetes_pod_label_app_kubernetes_io_name"
  rule {
    source_labels = ["__meta_kubernetes_pod_label_app_kubernetes_io_name"]
    action = "replace"
    target_label = "app"
  }

  // Label creation -  "job" field from "__meta_kubernetes_namespace" and "__meta_kubernetes_pod_container_name"
  // Concatenate values __meta_kubernetes_namespace/__meta_kubernetes_pod_container_name
  rule {
    source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_container_name"]
    action = "replace"
    target_label = "job"
    separator = "/"
    replacement = "$1"
  }

  // Label creation - "__path__" field from "__meta_kubernetes_pod_uid" and "__meta_kubernetes_pod_container_name"
  // Concatenate values __meta_kubernetes_pod_uid/__meta_kubernetes_pod_container_name.log
  rule {
    source_labels = ["__meta_kubernetes_pod_uid", "__meta_kubernetes_pod_container_name"]
    action = "replace"
    target_label = "__path__"
    separator = "/"
    replacement = "/var/log/pods/*$1/*.log"
  }

  // Label creation -  "container_runtime" field from "__meta_kubernetes_pod_container_id"
  rule {
    source_labels = ["__meta_kubernetes_pod_container_id"]
    action = "replace"
    target_label = "container_runtime"
    regex = `^(\S+):\/\/.+$`
    replacement = "$1"
  }
}

// loki.source.kubernetes tails logs from Kubernetes containers using the Kubernetes API.
loki.source.kubernetes "pod_logs" {
  targets    = discovery.relabel.pod_logs.output
  forward_to = [loki.process.pod_logs.receiver]
}

// loki.process receives log entries from other Loki components, applies one or more processing stages,
// and forwards the results to the list of receivers in the component's arguments.
loki.process "pod_logs" {
  stage.static_labels {
      values = {
        cluster = "<CLUSTER_NAME>",
      }
  }

  forward_to = [loki.write.<WRITE_COMPONENT_NAME>.receiver]
}

Replace the following values:

<CLUSTER_NAME>: The label for this specific Kubernetes cluster, such as production or us-east-1.
<WRITE_COMPONENT_NAME>: The name of your loki.write component, such as default.

Tip
Use raw strings delimited by backticks for regex values. Raw strings don’t process escape sequences, so patterns like \S and \d work without double escaping. Double-quoted strings require escaping, for example regex = "\\S" instead of regex = "\S". If you forget to escape, you get unknown escape sequence errors.

Note
Refer to [Limit to only Pods on the same node][discovery.kubernetes_samenode] for more information about restricting to Pods on the same node.

Kubernetes Cluster Events

Use the following components to collect Kubernetes cluster events:

loki.source.kubernetes_events: Tails events from the Kubernetes API.
loki.process: Modifies logs before sending them to the next component.
loki.write: Sends logs to the Loki endpoint. Configure this component in the Configure logs delivery section.

The following example collects Kubernetes cluster events:

// loki.source.kubernetes_events tails events from the Kubernetes API and converts them
// into log lines to forward to other Loki components.
loki.source.kubernetes_events "cluster_events" {
  job_name   = "integrations/kubernetes/eventhandler"
  log_format = "logfmt"
  forward_to = [
    loki.process.cluster_events.receiver,
  ]
}

// loki.process receives log entries from other loki components, applies one or more processing stages,
// and forwards the results to the list of receivers in the component's arguments.
loki.process "cluster_events" {
  forward_to = [loki.write.<WRITE_COMPONENT_NAME>.receiver]

  stage.static_labels {
    values = {
      cluster = "<CLUSTER_NAME>",
    }
  }

  stage.labels {
    values = {
      kubernetes_cluster_events = "job",
    }
  }
}

Replace the following values:

<CLUSTER_NAME>: The label for this specific Kubernetes cluster, such as production or us-east-1.
<WRITE_COMPONENT_NAME>: The name of your loki.write component, such as default.

Next steps

Collect Azure Event Hubs logs and forward them to Loki

Mon, 30 Mar 2026 15:47:22 +0000

Collect Azure Event Hubs logs and forward them to Loki

You can configure Alloy to collect logs from Azure Event Hubs and forward them to Loki. For more information about monitoring Azure resources in Grafana Cloud, refer to Monitor Microsoft Azure.

This topic describes how to:

Prepare your Azure environment with Workload Identity authentication.
Configure Azure Event Hubs and install Alloy.
Optionally extract labels from Azure resource logs.

Components used in this topic

Before you begin

Ensure you have the following:

Azure administrator access with Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator or User Access Administrator
Azure CLI installed and authenticated
kubectl installed and configured to access your AKS cluster
Helm installed

Prepare your Azure environment

Azure Event Hubs exposes different endpoints depending on the protocol:

Protocol	Port	When to use
Kafka	`9093` (TLS)	Applications send events using Kafka clients or Kafka Connect.
AMQP	`5671` (TLS)	Applications send events using Azure SDKs or AMQP client libraries.

Both protocols use the hostname <EVENTHUB_NAMESPACE>.servicebus.windows.net.

The loki.source.azure_event_hubs component uses the Kafka protocol, so the examples in this procedure use port 9093.

Use the Azure portal to create or reuse a resource group.
Create or reuse an Azure Kubernetes Service (AKS) cluster.
Enable Microsoft Entra Workload ID and OpenID Connect in your AKS cluster.
shell
```
az aks update \
    --resource-group <RESOURCE_GROUP> \
    --name <AKS_CLUSTER_NAME> \
    --enable-oidc-issuer \
    --enable-workload-identity
```
Replace the following:
- <RESOURCE_GROUP>: Your Azure resource group
- <AKS_CLUSTER_NAME>: The name of your AKS cluster
Retrieve the OIDC issuer URL for your cluster. You need this value when creating the federated credential.
shell
```
az aks show \
    --resource-group <RESOURCE_GROUP> \
    --name <AKS_CLUSTER_NAME> \
    --query "oidcIssuerProfile.issuerUrl" \
    --output tsv
```
Replace the following:
- <RESOURCE_GROUP>: Your Azure resource group
- <AKS_CLUSTER_NAME>: The name of your AKS cluster
Create a user-assigned managed identity.
shell
```
az identity create \
    --resource-group <RESOURCE_GROUP> \
    --name <MANAGED_IDENTITY_NAME>
```
Replace the following:
- <RESOURCE_GROUP>: Your Azure resource group
- <MANAGED_IDENTITY_NAME>: A name for your managed identity
Retrieve the client ID for your managed identity. You need this value for the ServiceAccount annotation and Alloy configuration.
shell
```
az identity show \
    --resource-group <RESOURCE_GROUP> \
    --name <MANAGED_IDENTITY_NAME> \
    --query clientId \
    --output tsv
```
Replace the following:
- <RESOURCE_GROUP>: Your Azure resource group
- <MANAGED_IDENTITY_NAME>: The name of your managed identity
Create a Kubernetes namespace for Alloy.
shell
```
kubectl create namespace alloy
```
Create a Kubernetes ServiceAccount with the workload identity annotation.
shell
```
kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: alloy
  namespace: alloy
  annotations:
    azure.workload.identity/client-id: "<CLIENT_ID>"
EOF
```
Replace the following:
- <CLIENT_ID>: The client ID from the previous step
Azure Workload Identity connects three components:

Kubernetes ServiceAccount : The ServiceAccount annotation azure.workload.identity/client-id specifies which managed identity the Pod can impersonate.

Federated credential : The federated credential on the managed identity trusts tokens from your AKS cluster’s OIDC issuer for a specific ServiceAccount (system:serviceaccount:<namespace>:<name>).

Runtime token exchange : When the Pod runs, AKS issues a token for the ServiceAccount. Azure validates this token against the federated credential and returns an access token for the managed identity. Alloy uses this token to authenticate to Event Hubs without a connection string.

If authentication fails, verify:
- The OIDC issuer URL matches your cluster
- The --subject value matches system:serviceaccount:<namespace>:<serviceaccount>
- The managed identity clientId matches the ServiceAccount annotation
- The --audiences value is api://AzureADTokenExchange

Create a federated identity credential to link your managed identity with the Kubernetes ServiceAccount.

az identity federated-credential create \
    --name alloy-federated-credential \
    --identity-name <MANAGED_IDENTITY_NAME> \
    --resource-group <RESOURCE_GROUP> \
    --issuer <OIDC_ISSUER_URL> \
    --subject system:serviceaccount:alloy:alloy \
    --audiences api://AzureADTokenExchange

Replace the following:

<RESOURCE_GROUP>: Your Azure resource group
<MANAGED_IDENTITY_NAME>: The name of your managed identity
<OIDC_ISSUER_URL>: The OIDC issuer URL

Configure Azure Event Hubs

Follow the steps to Set up Azure Event Hubs.
Assign the Azure Event Hubs Data Receiver role to your managed identity.

Get the managed identity principal ID:
shell
```
PRINCIPAL_ID=$(az identity show \
    --resource-group <RESOURCE_GROUP> \
    --name <MANAGED_IDENTITY_NAME> \
    --query principalId \
    --output tsv)
```
Assign the role at namespace scope for least-privilege access:
shell
```
az role assignment create \
    --assignee $PRINCIPAL_ID \
    --role "Azure Event Hubs Data Receiver" \
    --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.EventHub/namespaces/<EVENTHUB_NAMESPACE>
```
Replace the following:
- <MANAGED_IDENTITY_NAME>: The name of your managed identity
- <RESOURCE_GROUP>: Your Azure resource group
- <SUBSCRIPTION_ID>: Your Azure subscription ID
- <EVENTHUB_NAMESPACE>: The name of your Event Hub namespace
Assign the role at the smallest scope that meets your requirements:

Namespace scope (recommended) : Grants access only to the specific Event Hub namespace. Use this for least-privilege access.

Resource group or subscription scope : Grants access to all Event Hubs in the resource group or subscription. Use this only if Alloy must read from multiple namespaces.

Example resource group scope:
shell

az role assignment create \ --assignee $PRINCIPAL_ID \ --role "Azure Event Hubs Data Receiver" \ --scope /subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>
To verify the assignment:
shell

az role assignment list --assignee $PRINCIPAL_ID --scope <SCOPE>

Install Alloy

Add the Grafana Helm repository.

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

Retrieve the tenant ID for your Azure subscription.
shell
```
az account show --query tenantId --output tsv
```
Create a values.yaml file with the following configuration.

Warning
Don’t store sensitive credentials directly in values.yaml or commit them to version control. For production environments, use a Kubernetes Secret with secretKeyRef, or an external secret manager such as HashiCorp Vault, Azure Key Vault, or the External Secrets Operator.

The configuration uses port 9093 for the Azure Event Hubs Kafka-compatible endpoint. The loki.source.azure_event_hubs component in Alloy requires the Kafka-compatible endpoint and doesn’t support AMQP for this integration.

The authentication block uses OAuth 2.0 with Azure Workload Identity through the federated credential you created earlier. Kafka-compatible endpoints use SASL/OAUTHBEARER with Microsoft Entra ID tokens, so you don’t need an Event Hub connection string.
YAML
```
serviceAccount:
  create: false
  name: alloy

controller:
  type: statefulset
  replicas: 1
  podLabels:
    azure.workload.identity/use: "true"

alloy:
  extraEnv:
    - name: "AZURE_CLIENT_ID"
      value: "<CLIENT_ID>"
    - name: "AZURE_TENANT_ID"
      value: "<TENANT_ID>"
  configMap:
    content: |
      loki.source.azure_event_hubs "azure" {
        fully_qualified_namespace = "<EVENTHUB_NAMESPACE>.servicebus.windows.net:9093"
        event_hubs                = ["<EVENTHUB_NAME>"]

        authentication {
          mechanism = "oauth"
        }

        use_incoming_timestamp = true
        labels = {
          job = "integrations/azure_event_hubs",
        }
        forward_to = [loki.write.grafana_cloud.receiver]
      }

      loki.write "grafana_cloud" {
        endpoint {
          url = "<GRAFANA_CLOUD_LOKI_URL>"
          basic_auth {
            username = "<GRAFANA_CLOUD_LOKI_USERNAME>"
            password = "<GRAFANA_CLOUD_API_KEY>"
          }
        }
      }
```
Replace the following:
- <CLIENT_ID>: Your managed identity client ID
- <TENANT_ID>: Your Azure tenant ID
- <EVENTHUB_NAMESPACE>: Your Event Hub namespace name
- <EVENTHUB_NAME>: Your Event Hub name
- <GRAFANA_CLOUD_LOKI_URL>: Your Grafana Cloud Loki endpoint, such as https://logs-prod-us-central1.grafana.net/loki/api/v1/push
- <GRAFANA_CLOUD_LOKI_USERNAME>: Your Grafana Cloud Loki username
- <GRAFANA_CLOUD_API_KEY>: Your Grafana Cloud API key

Install Alloy using Helm.

helm install alloy grafana/alloy \
    --namespace alloy \
    -f values.yaml

Verify the installation

Check that the Alloy Pod is running.

kubectl get pods -n alloy

You should see output similar to:

NAME      READY   STATUS    RESTARTS   AGE
alloy-0   1/1     Running   0          1m

Check the Alloy logs for any errors.
shell
```
kubectl logs -n alloy -l app.kubernetes.io/name=alloy
```
- Verify authentication and connection:
  
  shell
  
  kubectl logs -n alloy -l app.kubernetes.io/name=alloy | grep -E -i "authenticated|connected|sasl"
- Push a test event to the Event Hub and confirm a matching log appears in Grafana Explore within approximately one minute.
- If errors occur, verify the role assignment:
  
  shell
  
  az role assignment list --assignee <PRINCIPAL_ID> --scope <SCOPE>
In Grafana Cloud, navigate to Explore and select your Loki data source to view the incoming logs.

Optional: Configure Alloy to extract labels from Azure Event Hubs

By default, the Alloy configuration doesn’t extract labels from the Event Hubs log lines.

You can configure Alloy to use loki.process to extract labels such as resourceId, category, and resourceGroup from Azure resource logs.

The loki.process component uses three stages to transform the logs:

stage.json: Parses the log line as JSON and extracts the resourceId and category fields.
stage.regex: Parses the resourceId to extract resource details like subscriptionId, resourceGroup, and resourceName.
stage.labels: Creates Loki labels from the extracted values for easier querying.

Update the alloy.configMap.content section in your values.yaml file with the following configuration.

When set to true, the use_incoming_timestamp setting uses the event’s timestamp, such as Event Hubs EnqueuedTimeUtc or a timestamp field in the payload. The default is false, which uses Alloy ingestion time. Keep the default if your events lack reliable timestamps.

alloy:
  configMap:
    content: |
      loki.source.azure_event_hubs "azure" {
        fully_qualified_namespace = "<EVENTHUB_NAMESPACE>.servicebus.windows.net:9093"
        event_hubs                = ["<EVENTHUB_NAME>"]

        authentication {
          mechanism = "oauth"
        }

        use_incoming_timestamp = true
        labels = {
          job = "integrations/azure_event_hubs",
        }
        forward_to = [loki.process.azure_logs.receiver]
      }

      loki.process "azure_logs" {
        stage.json {
          expressions = {
            resourceId = "resourceId",
            category   = "category",
          }
        }

        stage.regex {
          expression = "(?i)/subscriptions/(?P<subscriptionId>[^/]+)/resourcegroups/(?P<resourceGroup>[^/]+)/providers/(?P<providerNamespace>[^/]+)/(?P<resourceType>[^/]+)/(?P<resourceName>[^/]+)"
          source     = "resourceId"
        }

        stage.labels {
          values = {
            category       = "",
            resourceId     = "",
            resourceGroup  = "",
            service_name   = "resourceName",
          }
        }

        forward_to = [loki.write.grafana_cloud.receiver]
      }

      loki.write "grafana_cloud" {
        endpoint {
          url = "<GRAFANA_CLOUD_LOKI_URL>"
          basic_auth {
            username = "<GRAFANA_CLOUD_LOKI_USERNAME>"
            password = "<GRAFANA_CLOUD_API_KEY>"
          }
        }
      }

Replace the following:

<EVENTHUB_NAMESPACE>: Your Event Hub namespace name
<EVENTHUB_NAME>: Your Event Hub name
<GRAFANA_CLOUD_LOKI_URL>: Your Grafana Cloud Loki endpoint, such as https://logs-prod-us-central1.grafana.net/loki/api/v1/push
<GRAFANA_CLOUD_LOKI_USERNAME>: Your Grafana Cloud Loki username
<GRAFANA_CLOUD_API_KEY>: Your Grafana Cloud API key

After updating the configuration, upgrade the Helm release:

helm upgrade alloy grafana/alloy \
    --namespace alloy \
    -f values.yaml

Collect Prometheus metrics

Mon, 30 Mar 2026 15:47:22 +0000

Collect Prometheus metrics

You can configure Alloy to collect Prometheus metrics and forward them to any Prometheus-compatible database.

This topic describes how to:

Configure metrics delivery.
Collect metrics from Kubernetes Pods.

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with Prometheus.
Have a set of Prometheus exports or applications exposing Prometheus metrics that you want to collect metrics from.
Identify where to write collected metrics. Metrics can be written to Prometheus or Prometheus-compatible endpoints such as Grafana Mimir, Grafana Cloud, or Grafana Enterprise Metrics.
Be familiar with the concept of Components in Alloy.

Configure metrics delivery

Before components can collect Prometheus metrics, you must have a component responsible for writing those metrics somewhere.

The prometheus.remote_write component is responsible for delivering Prometheus metrics to one or more Prometheus-compatible endpoints. After a prometheus.remote_write component is defined, you can use other Alloy components to forward metrics to it.

To configure a prometheus.remote_write component for metrics delivery, complete the following steps:

Add the following prometheus.remote_write component to your configuration file.
Alloy
```
prometheus.remote_write "<LABEL>" {
  endpoint {
    url = "<PROMETHEUS_URL>"
  }
}
```
Replace the following:
- <LABEL>: The label for the component, such as default. The label you use must be unique across all prometheus.remote_write components in the same configuration file.
- <PROMETHEUS_URL> The full URL of the Prometheus-compatible endpoint where metrics are sent, such as https://prometheus-us-central1.grafana.net/api/v1/write for Prometheus or https://mimir-us-central1.grafana.net/api/v1/push/ for Mimir. The endpoint URL depends on the database you use.
If your endpoint requires basic authentication, paste the following inside the endpoint block.
Alloy
```
basic_auth {
  username = "<USERNAME>"
  password = "<PASSWORD>"
}
```
Replace the following:
- <USERNAME>: The basic authentication username.
- <PASSWORD>: The basic authentication password or API key.
If you have more than one endpoint to write metrics to, repeat the endpoint block for additional endpoints.

The following example demonstrates configuring prometheus.remote_write with multiple endpoints and mixed usage of basic authentication, and a prometheus.scrape component which forwards metrics to it.

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9009/api/prom/push"
  }

  endpoint {
    url = "https://prometheus-us-central1.grafana.net/api/prom/push"

    // Get basic authentication based on environment variables.
    basic_auth {
      username = sys.env("<REMOTE_WRITE_USERNAME>")
      password = sys.env("<REMOTE_WRITE_PASSWORD>")
    }
  }
}

prometheus.scrape "example" {
  // Collect metrics from the default listen address.
  targets = [{
    __address__ = "127.0.0.1:12345",
  }]

  forward_to = [prometheus.remote_write.default.receiver]
}

For more information on configuring metrics delivery, refer to prometheus.remote_write.

Collect metrics from Kubernetes Pods

Alloy can be configured to collect metrics from Kubernetes Pods by:

Discovering Kubernetes Pods to collect metrics from.
Collecting metrics from those discovered Pods.

To collect metrics from Kubernetes Pods, complete the following steps:

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Discover Kubernetes Pods:
1. Add the following discovery.kubernetes component to your configuration file to discover every Pod in the cluster across all Namespaces.
  Alloy
```
discovery.kubernetes "<DISCOVERY_LABEL>" {
  role = "pod"
}
```
  Replace the following
  - <DISCOVERY_LABEL>: The label for the component, such as pods. The label you use must be unique across all discovery.kubernetes components in the same configuration file.
  This generates one Prometheus target for every exposed port on every discovered Pod.
2. To limit the Namespaces that Pods are discovered in, add the following block inside the discovery.kubernetes component.
  Alloy
```
namespaces {
  own_namespace = true
  names         = [<NAMESPACE_NAMES>]
}
```
  Replace the following:
  - <NAMESPACE_NAMES>: A comma-delimited list of strings representing Namespaces to search. Each string must be wrapped in double quotes. For example, "default","kube-system".
  If you don’t want to search for Pods in the Namespace Alloy is running in, set own_namespace to false.
3. To use a field selector to limit the number of discovered Pods, add the following block inside the discovery.kubernetes component.
  Alloy
```
selectors {
  role  = "pod"
  field = "<FIELD_SELECTOR>"
}
```
  Replace the following:
  - <FIELD_SELECTOR>: The Kubernetes field selector to use, such as metadata.name=my-service. For more information on field selectors, refer to the Kubernetes documentation on Field Selectors.
  Create additional selectors blocks for each field selector you want to apply.
4. To use a label selector to limit the number of discovered Pods, add the following block inside the discovery.kubernetes component.
  Alloy
```
selectors {
  role  = "pod"
  label = "LABEL_SELECTOR"
}
```
  Replace the following:
  - <LABEL_SELECTOR>: The Kubernetes label selector, such as environment in (production, qa). For more information on label selectors, refer to the Kubernetes documentation on Labels and Selectors.
  Create additional selectors blocks for each label selector you want to apply.
Collect metrics from discovered Pods:
1. Add the following prometheus.scrape component to your configuration file.
  Alloy
```
prometheus.scrape "<SCRAPE_LABEL>" {
  targets    = discovery.kubernetes.<DISCOVERY_LABEL>.targets
  forward_to = [prometheus.remote_write.<REMOTE_WRITE_LABEL>.receiver]
}
```
  Replace the following:
  - <SCRAPE_LABEL>: The label for the component, such as pods. The label you use must be unique across all prometheus.scrape components in the same configuration file.
  - <DISCOVERY_LABEL>: The label for the discovery.kubernetes component.
  - <REMOTE_WRITE_LABEL>: The label for your prometheus.remote_write component.

The following example demonstrates configuring Alloy to collect metrics from running production Kubernetes Pods in the default Namespace.

discovery.kubernetes "pods" {
  role = "pod"

  namespaces {
    own_namespace = false

    names = ["default"]
  }

  selectors {
    role  = "pod"
    label = "environment in (production)"
  }
}

prometheus.scrape "pods" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9009/api/prom/push"
  }
}

For more information on configuring Kubernetes service delivery and collecting metrics, refer to discovery.kubernetes and prometheus.scrape.

Collect metrics from Kubernetes Services

You can configure Alloy to collect metrics from Kubernetes Services by:

Discovering Kubernetes Services to collect metrics from.
Collecting metrics from those discovered Services.

To collect metrics from Kubernetes Services, complete the following steps.

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Discover Kubernetes Services:
1. Add the following discovery.kubernetes component to your configuration file to discover every Services in the cluster across all Namespaces.
  Alloy
```
discovery.kubernetes "<DISCOVERY_LABEL>" {
  role = "service"
}
```
  Replace the following:
  - <DISCOVERY_LABEL>: A label for the component, such as services. The label you use must be unique across all discovery.kubernetes components in the same configuration file.
  This generates one Prometheus target for every exposed port on every discovered Service.
2. To limit the Namespaces that Services are discovered in, add the following block inside the discovery.kubernetes component.
  Alloy
```
namespaces {
  own_namespace = true
  names         = [<NAMESPACE_NAMES>]
}
```
  Replace the following:
  - <NAMESPACE_NAMES>: A comma-delimited list of strings representing Namespaces to search. Each string must be wrapped in double quotes. For example, "default","kube-system".
  If you don’t want to search for Services in the Namespace Alloy is running in, set own_namespace to false.
3. To use a field selector to limit the number of discovered Services, add the following block inside the discovery.kubernetes component.
  Alloy
```
selectors {
  role  = "service"
  field = "<FIELD_SELECTOR>"
}
```
  Replace the following:
  - <FIELD_SELECTOR>: The Kubernetes field selector, such as metadata.name=my-service. For more information on field selectors, refer to the Kubernetes documentation on Field Selectors.
  Create additional selectors blocks for each field selector you want to apply.
4. To use a label selector to limit the number of discovered Services, add the following block inside the discovery.kubernetes component.
  Alloy
```
selectors {
  role  = "service"
  label = "<LABEL_SELECTOR>"
}
```
  Replace the following:
  - <LABEL_SELECTOR>: The Kubernetes label selector, such as environment in (production, qa). For more information on label selectors, refer to the Kubernetes documentation on Labels and Selectors.
  Create additional selectors blocks for each label selector you want to apply.
Collect metrics from discovered Services:
1. Add the following prometheus.scrape component to your configuration file.
  Alloy
```
prometheus.scrape "<SCRAPE_LABEL>" {
  targets    = discovery.kubernetes.<DISCOVERY_LABEL>.targets
  forward_to = [prometheus.remote_write.<REMOTE_WRITE_LABEL>.receiver]
}
```
  Replace the following:
  - <SCRAPE_LABEL>: The label for the component, such as services. The label you use must be unique across all prometeus.scrape components in the same configuration file.
  - <DISCOVERY_LABEL>: The label for the discovery.kubernetes component.
  - <REMOTE_WRITE_LABEL>: The label for your prometheus.remote_write component.

The following example demonstrates configuring Alloy to collect metrics from running production Kubernetes Services in the default Namespace.

discovery.kubernetes "services" {
  role = "service"

  namespaces {
    own_namespace = false

    names = ["default"]
  }

  selectors {
    role  = "service"
    label = "environment in (production)"
  }
}

prometheus.scrape "services" {
  targets    = discovery.kubernetes.services.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9009/api/prom/push"
  }
}

For more information on configuring Kubernetes service delivery and collecting metrics, refer to discovery.kubernetes and prometheus.scrape.

Collect metrics from custom targets

You can configure Alloy to collect metrics from a custom set of targets without the need for service discovery.

To collect metrics from a custom set of targets, complete the following steps.

Follow Configure metrics delivery to ensure collected metrics can be written somewhere.
Add the following prometheus.scrape component to your configuration file:
Alloy
```
prometheus.scrape "<SCRAPE_LABEL>" {
  targets    = [<TARGET_LIST>]
  forward_to = [prometheus.remote_write.<REMOTE_WRITE_LABEL>.receiver]
}
```
Replace the following:
- _<SCRAPE_LABEL>: The label for the component, such as custom_targets. The label you use must be unique across all prometheus.scrape components in the same configuration file.
- <TARGET_LIST>: A comma-delimited list of Objects denoting the Prometheus target. Each object must conform to the following rules:
  - There must be an __address__ key denoting the HOST:PORT of the target to collect metrics from.
  - To explicitly specify which protocol to use, set the __scheme__ key to "http" or "https". If the __scheme__ key isn’t provided, the protocol to use is inherited by the settings of the prometheus.scrape component. The default is "http".
  - To explicitly specify which HTTP path to collect metrics from, set the __metrics_path__ key to the HTTP path to use. If the __metrics_path__ key isn’t provided, the path to use is inherited by the settings of the prometheus.scrape component. The default is "/metrics".
  - Add additional keys as desired to inject extra labels to collected metrics. Any label starting with two underscores (__) is dropped prior to scraping.
- <REMOTE_WRITE_LABEL>: The label for your prometheus.remote_write component.

The following example demonstrates configuring prometheus.scrape to collect metrics from a custom set of endpoints.

prometheus.scrape "custom_targets" {
  targets = [
    {
      __address__ = "prometheus:9090",
    },
    {
      __address__ = "mimir:8080",
      __scheme__  = "https",
    },
    {
      __address__      = "custom-application:80",
      __metrics_path__ = "/custom-metrics–path",
    },
    {
      __address__ = "alloy:12345",
      application = "alloy",
      environment = "production",
    },
  ]

  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://localhost:9009/api/prom/push"
  }
}

Collect OpenTelemetry data and forward it to any OpenTelemetry-compatible endpoint

Mon, 13 Apr 2026 06:53:30 +0000

Collect OpenTelemetry data and forward it to any OpenTelemetry-compatible endpoint

You can configure Alloy to collect OpenTelemetry-compatible data and forward it to any OpenTelemetry-compatible endpoint.

This topic describes how to:

Configure OpenTelemetry data delivery.
Configure batching.
Receive OpenTelemetry data over OTLP.

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
Have a set of OpenTelemetry applications ready to push telemetry data to Alloy.
Identify where Alloy writes received telemetry data.
Be familiar with the concept of Components in Alloy.

Configure an OpenTelemetry Protocol exporter

Before components can receive OpenTelemetry data, you must have a component responsible for exporting the OpenTelemetry data. An OpenTelemetry exporter component is responsible for writing (exporting) OpenTelemetry data to an external system.

In this task, you use the otelcol.exporter.otlp component to send OpenTelemetry data to a server using the OpenTelemetry Protocol (OTLP). After an exporter component is defined, you can use other Alloy components to forward data to it.

Tip
Refer to the list of available Components for the full list of otelcol.exporter components that you can use to export OpenTelemetry data.

To configure an otelcol.exporter.otlp component for exporting OpenTelemetry data using OTLP, complete the following steps:

Add the following otelcol.exporter.otlp component to your configuration file:
Alloy
```
otelcol.exporter.otlp "<EXPORTER_LABEL>" {
  client {
    endpoint = "<HOST>:<PORT>"
  }
}
```
Replace the following:
- <EXPORTER_LABEL>: The label for the component, such as default. The label you use must be unique across all otelcol.exporter.otlp components in the same configuration file.
- <HOST>: The hostname or IP address of the server to send OTLP requests to.
- <PORT>: The port of the server to send OTLP requests to.
If your server requires basic authentication, complete the following:
1. Add the following otelcol.auth.basic component to your configuration file:
  Alloy
```
otelcol.auth.basic "<BASIC_AUTH_LABEL>" {
  username = "<USERNAME>"
  password = "<PASSWORD>"
}
```
  Replace the following:
  - <BASIC_AUTH_LABEL>: The label for the component, such as default. The label you use must be unique across all otelcol.auth.basic components in the same configuration file.
  - <USERNAME>: The basic authentication username.
  - <PASSWORD>: The basic authentication password or API key.
2. Add the following line inside of the client block of your otelcol.exporter.otlp component:
  Alloy
```
auth = otelcol.auth.basic.<BASIC_AUTH_LABEL>.handler
```
  Replace the following:
  - <BASIC_AUTH_LABEL>: The label for the otelcol.auth.basic component.
If you have more than one server to export metrics to, create an otelcol.exporter.otlp component for each additional server.

Note
otelcol.exporter.otlp sends data using OTLP over gRPC (HTTP/2). To send to a server using HTTP/1.1, follow the preceding steps, but use the otelcol.exporter.otlphttp component instead.

The following example demonstrates configuring otelcol.exporter.otlp with authentication and a component that forwards data to it:

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.basic.credentials.handler
  }
}

otelcol.auth.basic "credentials" {
  // Retrieve credentials using environment variables.

  username = sys.env("BASIC_AUTH_USER")
  password = sys.env("API_KEY")
}

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

For more information on writing OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.exporter.otlp.

Configure batching

Production-ready Alloy configurations shouldn’t send OpenTelemetry data directly to an exporter for delivery. Instead, data is usually sent to one or more processor components that perform various transformations on the data.

Ensuring data is batched is a production-readiness step to improve data compression and reduce the number of outgoing network requests to external systems.

In this task, you configure an otelcol.processor.batch component to batch data before sending it to the exporter.

Note
Refer to the list of available Components for the full list of otelcol.processor components that you can use to process OpenTelemetry data. You can chain processors by having one processor send data to another processor.

To configure an otelcol.processor.batch component, complete the following steps:

Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.
Add the following otelcol.processor.batch component into your configuration file:
Alloy
```
otelcol.processor.batch "<PROCESSOR_LABEL>" {
  output {
    metrics = [otelcol.exporter.otlp.<EXPORTER_LABEL>.input]
    logs    = [otelcol.exporter.otlp.<EXPORTER_LABEL>.input]
    traces  = [otelcol.exporter.otlp.<EXPORTER_LABEL>.input]
  }
}
```
Replace the following:
- <PROCESSOR_LABEL>: The label for the component, such as default. The label you use must be unique across all otelcol.processor.batch components in the same configuration file.
- <EXPORTER_LABEL>: The label for your otelcol.exporter.otlp component.
1. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].
2. To send batched data to another processor, replace the components in the output list with the processor components to use.

The following example demonstrates configuring a sequence of otelcol.processor components before being exported.

otelcol.processor.memory_limiter "default" {
  check_interval = "1s"
  limit          = "1GiB"

  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on configuring OpenTelemetry data batching, refer to otelcol.processor.batch.

Configure an OpenTelemetry Protocol receiver

You can configure Alloy to receive OpenTelemetry metrics, logs, and traces. An OpenTelemetry receiver component is responsible for receiving OpenTelemetry data from an external system.

In this task, you use the otelcol.receiver.otlp component to receive OpenTelemetry data over the network using the OpenTelemetry Protocol (OTLP). You can configure a receiver component to forward received data to other Alloy components.

Refer to the list of available Components for the full list of otelcol.receiver components that you can use to receive OpenTelemetry-compatible data.

To configure an otelcol.receiver.otlp component for receiving OTLP data, complete the following steps:

Follow Configure an OpenTelemetry Protocol exporter to ensure received data can be written to an external system.
Optional: Follow Configure batching to improve compression and reduce the total amount of network requests.
Add the following otelcol.receiver.otlp component to your configuration file.
Alloy
```
otelcol.receiver.otlp "<LABEL>" {
  output {
    metrics = [<COMPONENT_INPUT_LIST>]
    logs    = [<COMPONENT_INPUT_LIST>]
    traces  = [<COMPONENT_INPUT_LIST>]
  }
}
```
Replace the following:
- <LABEL>: The label for the component, such as default. The label you use must be unique across all otelcol.receiver.otlp components in the same configuration file.
- <COMPONENT_INPUT_LIST>: A comma-delimited list of component inputs to forward received data to. For example, to send data to a batch processor component, use otelcol.processor.batch.PROCESSOR_LABEL.input. To send data directly to an exporter component, use otelcol.exporter.otlp.EXPORTER_LABEL.input.
1. To allow applications to send OTLP data over gRPC on port 4317, add the following to your otelcol.receiver.otlp component.
  Alloy
```
grpc {
  endpoint = "<HOST>:4317"
}
```
  Replace the following:
  - <HOST>: A host to listen to traffic on. Use a narrowly scoped listen address whenever possible. To listen on all network interfaces, replace <HOST> with 0.0.0.0.
2. To allow applications to send OTLP data over HTTP/1.1 on port 4318, add the following to your otelcol.receiver.otlp component.
  Alloy
```
http {
  endpoint = "<HOST>:4318"
}
```
  Replace the following:
  - <HOST>: The host to listen to traffic on. Use a narrowly scoped listen address whenever possible. To listen on all network interfaces, replace <HOST> with 0.0.0.0.
3. To disable one of the telemetry types, set the relevant type in the output block to the empty list, such as metrics = [].

The following example demonstrates configuring otelcol.receiver.otlp and sending it to an exporter:

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

For more information on receiving OpenTelemetry data using the OpenTelemetry Protocol, refer to otelcol.receiver.otlp.

Collect OpenTelemetry data and forward to Grafana

Mon, 30 Mar 2026 15:47:22 +0000

Collect OpenTelemetry data and forward to Grafana

You can configure Alloy to collect OpenTelemetry-compatible data and forward it to the Grafana stack.

This topic describes how to:

Configure Alloy to send your data to Loki.
Configure Alloy to send your data to Tempo.
Configure Alloy to send your data to Mimir or Prometheus Remote Write.

Components used in this topic

Before you begin

Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
Have a set of OpenTelemetry applications ready to push telemetry data to Alloy.
Identify where Alloy writes received telemetry data.
Be familiar with the concept of Components in Alloy.
Complete the Collect open telemetry data task.

The pipeline

You can start with the Alloy configuration you created in the Collect open telemetry data task.

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
  }
}

The pipeline looks like this:

Metrics, Logs, Traces: OTLP Receiver → Batch Processor → OTLP HTTP Exporter

Grafana Cloud

Grafana Cloud provides OTLP Endpoints that you can use directly from within Alloy.

You can find the OTLP connection details from the OpenTelemetry Details page in the Grafana Cloud Portal.

You must update the configuration file as follows:

otelcol.auth.basic "default" {
  username = "<ACCOUNT ID>"
  password = "<API TOKEN>"
}

otelcol.exporter.otlphttp "default" {
  client {
    endpoint = "<OTLP_ENDPOINT>"
    auth     = otelcol.auth.basic.default.handler
  }
}

Replace the following:

<ACCOUNT ID>: Your Grafana Cloud account ID.
<API TOKEN>: Your Grafana Cloud API token.
<OTLP_ENDPOINT>: Your OTLP endpoint.

This configuration uses the credentials stored in otelcol.auth.basic "default" to authenticate against the Grafana Cloud OTLP endpoints, and you should start to see your data arrive.

Note
For Grafana Cloud, use otelcol.exporter.otlphttp which sends data over HTTP/HTTPS. Use otelcol.exporter.otlp for gRPC endpoints when working with other platforms.

Other platforms: Grafana Enterprise and Grafana Open Source

You can implement the following pipelines to send your data to Loki, Tempo, and Mimir or Prometheus.

Metrics: OTLP Receiver → Batch Processor → Prometheus Exporter → Prometheus Remote Write
Logs: OTLP Receiver → Batch Processor → Loki Exporter → Loki Write  
Traces: OTLP Receiver → Batch Processor → OTLP Exporter (gRPC/HTTP)

Grafana Loki

Grafana Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. Similar to Prometheus, to send from OTLP to Loki, you can configure pass-through from the otelcol.exporter.loki component to loki.write component.

otelcol.exporter.loki "default" {
  forward_to = [loki.write.default.receiver]
}

loki.write "default" {
  endpoint {
    url = "http://loki-endpoint:8080/loki/api/v1/push"
        }
}

To use Loki with basic-auth, which Grafana Cloud Logs requires, you must configure the loki.write component. You can get the Loki configuration from the Loki Details page in the Grafana Cloud Portal.

otelcol.exporter.loki "grafana_cloud_logs" {
  forward_to = [loki.write.grafana_cloud_logs.receiver]
}

loki.write "grafana_cloud_logs" {
  endpoint {
    url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"

    basic_auth {
      username = "5252"
      password = sys.env("GRAFANA_CLOUD_API_KEY")
    }
  }
}

Grafana Tempo

Grafana Tempo is an open source, scalable distributed tracing backend. Tempo can ingest OTLP directly, and you can use the OTLP exporter to send the traces to Tempo.

otelcol.exporter.otlp "default" {
  client {
    endpoint = "tempo-server:4317"
  }
}

To use Tempo with basic-auth, which Grafana Cloud Traces requires, you must use the otelcol.auth.basic component. You can get the Tempo configuration from the Tempo Details page in the Grafana Cloud Portal.

otelcol.exporter.otlphttp "grafana_cloud_traces" {
  client {
    endpoint = "https://tempo-us-central1.grafana.net:443"
    auth     = otelcol.auth.basic.grafana_cloud_traces.handler
  }
}

otelcol.auth.basic "grafana_cloud_traces" {
  username = "4094"
  password = sys.env("GRAFANA_CLOUD_API_KEY")
}

Grafana Mimir or Prometheus Remote Write

Prometheus Remote Write is a popular metrics transmission protocol supported by most metrics systems, including Grafana Mimir and Grafana Cloud. To send from OTLP to a Prometheus compatible remote_write endpoint, you can configure pass-through from the otelcol.exporter.prometheus to the prometheus.remote_write component. The Prometheus remote write component in Alloy is a robust protocol implementation, including a Write Ahead Log (WAL) for resiliency.

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "http://prometheus:9090/api/v1/write"
  }
}

To use Prometheus with basic-auth, which Grafana Cloud Metrics requires, you must configure the prometheus.remote_write component. You can get the Prometheus configuration from the Prometheus Details page in the Grafana Cloud Portal.

otelcol.exporter.prometheus "grafana_cloud_metrics" {
        forward_to = [prometheus.remote_write.grafana_cloud_metrics.receiver]
    }

prometheus.remote_write "grafana_cloud_metrics" {
    endpoint {
        url = "https://prometheus-us-central1.grafana.net/api/prom/push"

        basic_auth {
            username = "12690"
            password = sys.env("GRAFANA_CLOUD_API_KEY")
        }
    }
}

Put it all together

Instead of referencing otelcol.exporter.otlp.default.input in the output of otelcol.processor.batch, you need to reference the three exporters you set up. For Grafana Cloud, use otelcol.exporter.otlphttp for traces to send data over HTTPS. The final configuration becomes:

otelcol.receiver.otlp "example" {
  grpc {
    endpoint = "127.0.0.1:4317"
  }

  http {
    endpoint = "127.0.0.1:4318"
  }

  output {
    metrics = [otelcol.processor.batch.example.input]
    logs    = [otelcol.processor.batch.example.input]
    traces  = [otelcol.processor.batch.example.input]
  }
}

otelcol.processor.batch "example" {
  output {
    metrics = [otelcol.exporter.prometheus.grafana_cloud_metrics.input]
    logs    = [otelcol.exporter.loki.grafana_cloud_logs.input]
    traces  = [otelcol.exporter.otlphttp.grafana_cloud_traces.input]
  }
}

otelcol.exporter.otlphttp "grafana_cloud_traces" {
  client {
    endpoint = "https://tempo-us-central1.grafana.net:443"
    auth     = otelcol.auth.basic.grafana_cloud_traces.handler
  }
}

otelcol.auth.basic "grafana_cloud_traces" {
  username = "4094"
  password = sys.env("GRAFANA_CLOUD_API_KEY")
}

otelcol.exporter.prometheus "grafana_cloud_metrics" {
        forward_to = [prometheus.remote_write.grafana_cloud_metrics.receiver]
    }

prometheus.remote_write "grafana_cloud_metrics" {
    endpoint {
        url = "https://prometheus-us-central1.grafana.net/api/prom/push"

        basic_auth {
            username = "12690"
            password = sys.env("GRAFANA_CLOUD_API_KEY")
        }
    }
}

otelcol.exporter.loki "grafana_cloud_logs" {
  forward_to = [loki.write.grafana_cloud_logs.receiver]
}

loki.write "grafana_cloud_logs" {
  endpoint {
    url = "https://logs-prod-us-central1.grafana.net/loki/api/v1/push"

    basic_auth {
      username = "5252"
      password = sys.env("GRAFANA_CLOUD_API_KEY")
    }
  }
}

Running Alloy gives you the following:

./alloy run alloy-config.alloy
./alloy run alloy-config.alloy
ts=2023-05-09T09:37:15.300959Z level=info msg="running usage stats reporter"
ts=2023-05-09T09:37:15.300958Z level=info msg="now listening for http traffic" addr=127.0.0.1:12345
ts=2023-05-09T09:37:15.301104Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="starting complete graph evaluation"
ts=2023-05-09T09:37:15.301307Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=loki.write.grafana_cloud_logs duration=188.209µs
ts=2023-05-09T09:37:15.301334Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.loki.grafana_cloud_logs duration=18.791µs
ts=2023-05-09T09:37:15.303138Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="replaying WAL, this may take a while" dir=data-alloy/prometheus.remote_write.grafana_cloud_metrics/wal
ts=2023-05-09T09:37:15.303257Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="WAL segment loaded" segment=0 maxSegment=1
ts=2023-05-09T09:37:15.303302Z component=prometheus.remote_write.grafana_cloud_metrics level=info subcomponent=wal msg="WAL segment loaded" segment=1 maxSegment=1
ts=2023-05-09T09:37:15.303507Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting WAL watcher" queue=7f623a
ts=2023-05-09T09:37:15.303515Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Starting scraped metadata watcher"
ts=2023-05-09T09:37:15.303522Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=prometheus.remote_write.grafana_cloud_metrics duration=2.181958ms
ts=2023-05-09T09:37:15.303557Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.prometheus.grafana_cloud_metrics duration=30.083µs
ts=2023-05-09T09:37:15.303611Z component=prometheus.remote_write.grafana_cloud_metrics subcomponent=rw level=info remote_name=7f623a url=https://prometheus-us-central1.grafana.net/api/prom/push msg="Replaying WAL" queue=7f623a
ts=2023-05-09T09:37:15.303618Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.auth.basic.grafana_cloud_traces duration=52.5µs
ts=2023-05-09T09:37:15.303694Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.exporter.otlphttp.grafana_cloud_traces duration=70.375µs
ts=2023-05-09T09:37:15.303782Z component=otelcol.processor.memory_limiter.default level=info msg="Memory limiter configured" limit_mib=150 spike_limit_mib=30 check_interval=1s
ts=2023-05-09T09:37:15.303802Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.memory_limiter.default duration=100.334µs
ts=2023-05-09T09:37:15.303853Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.processor.batch.default duration=44.75µs
ts=2023-05-09T09:37:15.303948Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=otelcol.receiver.otlp.default duration=87.333µs
ts=2023-05-09T09:37:15.303968Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=tracing duration=10.792µs
ts=2023-05-09T09:37:15.303981Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished node evaluation" node_id=logging duration=9µs
ts=2023-05-09T09:37:15.303987Z level=info trace_id=6466516c9e1a556422df7a84c0ade6b0 msg="finished complete graph evaluation" duration=2.960333ms
ts=2023-05-09T09:37:15.304Z level=info msg="scheduling loaded components"
ts=2023-05-09T09:37:15.304109Z component=otelcol.receiver.otlp.default level=info msg="Starting GRPC server" endpoint=0.0.0.0:4317
ts=2023-05-09T09:37:15.304234Z component=otelcol.receiver.otlp.default level=info msg="Starting HTTP server" endpoint=0.0.0.0:4318

You can check the pipeline graphically by visiting http://localhost:12345/graph

Collect Amazon Elastic Container Service or AWS Fargate OpenTelemetry data

Mon, 30 Mar 2026 15:47:22 +0000

Collect Amazon Elastic Container Service or AWS Fargate OpenTelemetry data

You can configure Grafana Alloy or AWS ADOT to collect OpenTelemetry-compatible data from Amazon Elastic Container Service (ECS) or AWS Fargate and forward it to any OpenTelemetry-compatible endpoint.

Metrics are available from various sources including ECS itself, the ECS instances when using EC2, X-Ray, and your own application. You can also collect logs and traces from your applications instrumented for Prometheus or OTLP.

Before you begin

Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
Have an available Amazon ECS or AWS Fargate deployment.
Identify where Alloy writes received telemetry data.
Familiarize yourself with the concept of Components in Alloy.

Collect task and container metrics

In this configuration, you add an OpenTelemetry Collector to the task running your application, and it uses the ECS Metadata Endpoint to gather task and container metrics in your cluster.

You can choose between two collector implementations:

You can use ADOT, the AWS OpenTelemetry collector. ADOT has native support for scraping task and container metrics. ADOT comes with default configurations that you can select in the task definition.
You can use Alloy with either the otelcol.receiver.awsecscontainermetrics component or the Prometheus ECS exporter sidecar.

Configure ADOT

If you use ADOT as a collector, add a container to your task definition and use a custom configuration you define in your AWS Systems Manager Parameter Store.

You can find sample OpenTelemetry configuration files in the AWS Observability repository. You can use these samples as a starting point and add the appropriate exporter configuration to send metrics to a Prometheus or OpenTelemetry endpoint.

Use ecs-default-config to consume StatsD metrics, OTLP metrics and traces, and AWS X-Ray SDK traces.
Use otel-task-metrics-config to consume StatsD, OTLP, AWS X-Ray, and Container Resource utilization metrics.

Read otel-prometheus to find out how to set the Prometheus remote write. The example uses AWS managed Prometheus.

Complete the following steps to create a sample task. Refer to the ADOT doc for more information.

Create a Parameter Store entry to hold the collector configuration file.
1. Open the AWS Console.
2. In the AWS Console, choose Parameter Store.
3. Choose Create parameter.
4. Create a parameter with the following values:
  - Name: collector-config
  - Tier: Standard
  - Type: String
  - Data type: Text
  - Value: Copy and paste your custom OpenTelemetry configuration file.
Download the ECS Fargate or ECS EC2 task definition template from GitHub.
Edit the task definition template and add the following parameters.
- {{region}}: The region to send the data to.
- {{ecsTaskRoleArn}}: The AWSOTTaskRole ARN.
- {{ecsTaskExecutionRoleArn}}: The AWSOTTaskExecutionRole ARN.
- Add an environment variable named AOT_CONFIG_CONTENT. Select ValueFrom to tell ECS to get the value from the Parameter Store, and set the value to collector-config.
Follow the ECS Fargate setup instructions to create a task definition using the template.

Configure Alloy

Alloy offers two approaches for collecting ECS task and container metrics:

Native receiver: Use the otelcol.receiver.awsecscontainermetrics component to read the ECS Task Metadata Endpoint V4 directly. This approach is simpler but requires enabling experimental features.
Prometheus exporter: Use the Prometheus ECS exporter as a sidecar container and scrape metrics with prometheus.scrape. This approach requires an additional container.

Native receiver

The otelcol.receiver.awsecscontainermetrics component reads AWS ECS task- and container-level metadata, and resource usage metrics such as CPU, memory, network, and disk, and forwards them to other otelcol.* components.

Note
The otelcol.receiver.awsecscontainermetrics component is experimental. You must set the --stability.level=experimental flag to use it. Refer to Permitted stability levels for more information.

Use the following as a starting point for your Alloy configuration:

otelcol.receiver.awsecscontainermetrics "default" {
  collection_interval = "60s"

  output {
    metrics = [otelcol.exporter.otlphttp.default.input]
  }
}

otelcol.exporter.otlphttp "default" {
  client {
    endpoint = sys.env("OTLP_ENDPOINT")

    auth = otelcol.auth.basic.default.handler
  }
}

otelcol.auth.basic "default" {
  client_auth {
    username = sys.env("OTLP_USERNAME")
    password = sys.env("OTLP_PASSWORD")
  }
}

This configuration collects ECS task and container metrics every 60 seconds and exports them to an OTLP endpoint.

Prometheus exporter

You don’t need to enable experimental stability in Alloy to use the ECS exporter as a sidecar container.

Use the following as a starting point for your Alloy configuration:

prometheus.scrape "ecs" {
  targets    = [{"__address__" = "localhost:9779"}]
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = sys.env("PROMETHEUS_REMOTE_WRITE_URL")

    basic_auth {
      username = sys.env("PROMETHEUS_USERNAME")
      password = sys.env("PROMETHEUS_PASSWORD")
    }
  }
}

This configuration scrapes metrics from the ECS exporter running on port 9779 and exports them to a Prometheus endpoint.

Deploy the task

Complete the following steps to create a sample task.

Create a Parameter Store entry to hold the collector configuration file.
1. Open the AWS Console.
2. In the AWS Console, choose Parameter Store.
3. Choose Create parameter.
4. Create a parameter with the following values:
  - Name: collector-config
  - Tier: Standard
  - Type: String
  - Data type: Text
  - Value: Copy and paste your custom Alloy configuration file.
Download the ECS Fargate or ECS EC2 task definition template from GitHub.
Edit the task definition template and add the following parameters.
- {{region}}: The region to send the data to.
- {{ecsTaskRoleArn}}: The AWSOTTaskRole ARN.
- {{ecsTaskExecutionRoleArn}}: The AWSOTTaskExecutionRole ARN.
- Set the container image to grafana/alloy:<VERSION>, for example grafana/alloy:latest or a specific version such as grafana/alloy:v1.8.0.
- Add a custom environment variable named ALLOY_CONFIG_CONTENT. Select ValueFrom to tell ECS to get the value from the Parameter Store, and set the value to collector-config. Alloy doesn’t read this variable directly, but you use it with the command below to pass the configuration.
- Add environment variables for your endpoint.
  - For the native receiver:
    - OTLP_ENDPOINT
    - OTLP_USERNAME
    - OTLP_PASSWORD - For increased security, create a password in AWS Secrets Manager and reference the ARN of the secret in the ValueFrom field.
  - For the Prometheus exporter:
    - PROMETHEUS_REMOTE_WRITE_URL
    - PROMETHEUS_USERNAME
    - PROMETHEUS_PASSWORD - For increased security, create a password in AWS Secrets Manager and reference the ARN of the secret in the ValueFrom field.
- In the Docker configuration, change the Entrypoint to /bin/sh,-c. This allows the command to use shell features like redirection and chaining.
- {{command}}: For the native receiver, use "printenv ALLOY_CONFIG_CONTENT > /tmp/config_file && exec /bin/alloy run --stability.level=experimental --server.http.listen-addr=0.0.0.0:12345 /tmp/config_file". The --stability.level=experimental flag enables the use of the otelcol.receiver.awsecscontainermetrics component.
  
  For the Prometheus exporter, use "printenv ALLOY_CONFIG_CONTENT > /tmp/config_file && exec /bin/alloy run --server.http.listen-addr=0.0.0.0:12345 /tmp/config_file".
  
  Make sure you include the double quotes around the command.
- Prometheus exporter only: Add the Prometheus ECS exporter as a sidecar container:
  1. Add a container to the task.
  2. Set the container name to ecs-exporter.
  3. Set the image to a pinned version of the exporter, for example quay.io/prometheuscommunity/ecs-exporter:v0.1.1. Check the ecs_exporter releases for the latest stable version. You should avoid using the latest tag in production.
  4. Add tcp/9779 as a port mapping.
Follow the ECS Fargate setup instructions to create a task definition using the template.

Collect EC2 instance metrics

For ECS Clusters running on EC2, you can collect instance metrics by using AWS ADOT or Alloy in a separate ECS task deployed as a daemon.

Collect metrics with Alloy

You can follow the steps described in Configure Alloy to create another task, with the following changes:

If you’re using the awsecscontainermetrics receiver, you don’t need to add the ecs-exporter sidecar for EC2 instance metrics.
Run the task as a daemon so it automatically runs one instance per node in your cluster.
Update your Alloy configuration to collect metrics from the instance. The configuration varies depending on the type of EC2 node. Refer to the collect documentation for details.

Collect metrics with ADOT

The approach described in the AWS OpenTelemetry documentation uses the awscontainerinsightreceiver receiver from OpenTelemetry. ADOT includes this receiver.

You need to use a custom configuration Parameter Store entry based on the sample configuration file to route the telemetry to your final destination.

Collect application telemetry

To collect metrics and traces emitted by your application, use the OTLP endpoints exposed by the collector sidecar container regardless of the collector implementation. Specify localhost as the host name, which is the default for most instrumentation libraries and agents.

For Prometheus endpoints, add a scrape job to the ADOT or Alloy configuration, and use localhost, service port, and endpoint path.

Collect Logs

The easiest way to collect application logs in ECS is to use the AWS FireLens log driver. Depending on your use case, you can forward your logs to the collector container in your task using the Fluent Bit plugin for OpenTelemetry or using the Fluent Bit Loki plugin. You can also send everything directly to your final destination.