Documentation Grafana Alloy Reference Components otelcol otelcol.processor.k8sattributes

otelcol.processor.k8sattributes

otelcol.processor.k8sattributes accepts telemetry data from other otelcol components and adds Kubernetes metadata to the resource attributes of spans, logs, or metrics.

You can specify multiple otelcol.processor.k8sattributes components by giving them different labels.

Usage

alloy Copy

otelcol.processor.k8sattributes "<LABEL>" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

You can use the following arguments with otelcol.processor.k8sattributes:

The supported values for auth_type are:

• none: No authentication is required.
• serviceAccount: Use the built-in service account that Kubernetes automatically provisions for each Pod.
• kubeConfig: Use local credentials like those used by kubectl.
• tls: Use client TLS authentication.

Setting passthrough to true enables the “passthrough mode” of otelcol.processor.k8sattributes:

• Only a k8s.pod.ip resource attribute will be added.
• No other metadata will be added.
• The Kubernetes API won’t be accessed.
• To correctly detect the Pod IPs, Alloy must receive spans directly from services.
• The passthrough setting is useful when configuring Alloy as a Kubernetes Deployment.

A Alloy running as a Deployment can’t detect the IP addresses of pods generating telemetry data without any of the well-known IP attributes. If the Deployment Alloy receives telemetry from Alloys deployed as DaemonSet, then some of those attributes might be missing. As a workaround, you can configure the DaemonSet Alloys with passthrough set to true.

By default, otelcol.processor.k8sattributes is ready as soon as it starts, even if no metadata has been fetched yet. If telemetry is sent to this processor before the metadata is synced, there will be no metadata to enrich the telemetry with.

To wait for the metadata to be synced before otelcol.processor.k8sattributes is ready, set the wait_for_metadata option to true. Then, the processor won’t be ready until the metadata is fully synced. As a result, the start-up of Alloy will be blocked. If the metadata can’t be synced by the time the wait_for_metadata_timeout duration is reached, otelcol.processor.k8sattributes will become unhealthy and fail to start.

If otelcol.processor.k8sattributes is unhealthy, other Alloy components will still be able to start. However, they may be unable to send telemetry to otelcol.processor.k8sattributes.

Blocks

You can use the following blocks with otelcol.processor.k8sattributes:

The > symbol indicates deeper levels of nesting. For example, extract > annotation refers to an annotation block defined inside an extract block.

output

debug_metrics

exclude

The exclude block configures which pods to exclude from the processor.

pod

The pod block configures a Pod to be excluded from the processor.

The following attributes are supported:

extract

The extract block configures which metadata, annotations, and labels to extract from the Pod.

The following attributes are supported:

The supported metadata keys are:

• container.id
• container.image.name
• container.image.tag
• k8s.container.name
• k8s.cronjob.name
• k8s.daemonset.name
• k8s.daemonset.uid
• k8s.deployment.name
• k8s.job.name
• k8s.job.uid
• k8s.namespace.name
• k8s.node.name
• k8s.pod.name
• k8s.pod.start_time
• k8s.pod.uid
• k8s.replicaset.name
• k8s.replicaset.uid
• k8s.statefulset.name
• k8s.statefulset.uid
• service.instance.id
• service.name
• service.namespace
• service.version

The service.* metadata are calculated following the OpenTelemetry semantic conventions.

By default, if metadata isn’t specified, the following fields are extracted and added to spans, metrics, and logs as resource attributes:

• container.image.name (requires one of the following additional attributes to be set: container.id or k8s.container.name)
• container.image.tag (requires one of the following additional attributes to be set: container.id or k8s.container.name)
• k8s.container.name (requires an additional attribute to be set: container.id)
• k8s.deployment.name (if the Pod is controlled by a deployment)
• k8s.namespace.name
• k8s.node.name
• k8s.pod.name
• k8s.pod.start_time
• k8s.pod.uid

When otel_annotations is set to true, annotations such as resource.opentelemetry.io/exampleResource will be translated to the exampleResource resource attribute, etc.

annotation

The annotation block configures how to extract Kubernetes annotations.

The following attributes are supported:

Name	Type	Description	Default	Required
from	string	The source of the labels or annotations. Allowed values are pod, namespace, and node.	pod	no
key_regex	string	A regular expression used to extract a key that matches the regular expression.	""	no
key	string	The annotation or label name. This key must exactly match an annotation or label name.	""	no
tag_name	string	The name of the resource attribute added to logs, metrics, or spans.	""	no

When you don’t specify the tag_name, a default tag name is used with the format:

• k8s.pod.annotations.<annotation key>
• k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your Pod spec contains the following labels:

• app.kubernetes.io/component: mysql
• app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

alloy Copy

extract {
  label {
    from = "pod"
    key_regex = "kubernetes.io/(.*)"
    tag_name  = "$1"
  }
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to "pod", "namespace", "node", or "deployment".

label extract

The label block configures how to extract Kubernetes labels.

The following attributes are supported:

Name	Type	Description	Default	Required
from	string	The source of the labels or annotations. Allowed values are pod, namespace, and node.	pod	no
key_regex	string	A regular expression used to extract a key that matches the regular expression.	""	no
key	string	The annotation or label name. This key must exactly match an annotation or label name.	""	no
tag_name	string	The name of the resource attribute added to logs, metrics, or spans.	""	no

When you don’t specify the tag_name, a default tag name is used with the format:

• k8s.pod.annotations.<annotation key>
• k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your Pod spec contains the following labels:

• app.kubernetes.io/component: mysql
• app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

alloy Copy

extract {
  label {
    from = "pod"
    key_regex = "kubernetes.io/(.*)"
    tag_name  = "$1"
  }
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to "pod", "namespace", "node", or "deployment".

filter

The filter block configures which nodes to get data from and which fields and labels to fetch.

The following attributes are supported:

If node is specified, then any pods not running on the specified node will be ignored by otelcol.processor.k8sattributes.

field

The field block allows you to filter pods by generic Kubernetes fields.

label filter

The label block allows you to filter pods by generic Kubernetes labels.

pod_association

The pod_association block configures rules on how to associate logs/traces/metrics to pods.

The pod_association block doesn’t support any arguments and is configured fully through child blocks.

The pod_association block can be repeated multiple times, to configure additional rules.

Example

alloy Copy

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.ip"
    }
}

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.uid"
    }
    source {
        from = "connection"
    }
}

source

The source block configures a Pod association rule. This is used by the k8sattributes processor to determine the Pod associated with a telemetry signal.

When multiple source blocks are specified inside a pod_association block, both source blocks has to match for the Pod to be associated with the telemetry signal.

The following attributes are supported:

Exported fields

The following fields are exported and can be referenced by other components:

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.k8sattributes is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.k8sattributes doesn’t expose any component-specific debug information.

Examples

Basic usage

In most cases, this is enough to get started. It’ll add these resource attributes to all logs, metrics, and traces:

• k8s.deployment.name
• k8s.namespace.name
• k8s.node.name
• k8s.pod.name
• k8s.pod.start_time
• k8s.pod.uid

Example:

alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = sys.env("<OTLP_ENDPOINT>")
  }
}

Add additional metadata and labels

alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from      = "pod"
      key_regex = "(.*)/(.*)"
      tag_name  = "$1.$2"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.deployment.name",
      "k8s.statefulset.name",
      "k8s.daemonset.name",
      "k8s.cronjob.name",
      "k8s.job.name",
      "k8s.node.name",
      "k8s.pod.name",
      "k8s.pod.uid",
      "k8s.pod.start_time",
    ]

    otel_annotations = true
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = sys.env("<OTLP_ENDPOINT>")
  }
}

Add Kubernetes metadata to Prometheus metrics

otelcol.processor.k8sattributes adds metadata to metrics signals in the form of resource attributes. To display the metadata as labels of Prometheus metrics, the OTLP attributes must be converted from resource attributes to datapoint attributes. One way to do this is by using an otelcol.processor.transform component.

alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from = "pod"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.pod.name",
    ]
  }

  output {
    metrics = [otelcol.processor.transform.add_kube_attrs.input]
  }
}

otelcol.processor.transform "add_kube_attrs" {
  error_mode = "ignore"

  metric_statements {
    context = "datapoint"
    statements = [
      "set(attributes[\"k8s.pod.name\"], resource.attributes[\"k8s.pod.name\"])",
      "set(attributes[\"k8s.namespace.name\"], resource.attributes[\"k8s.namespace.name\"])",
    ]
  }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Compatible components

otelcol.processor.k8sattributes can accept arguments from the following components:

• Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.k8sattributes has exports that can be consumed by the following components:

• Components that consume OpenTelemetry otelcol.Consumer