Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Alloy Reference Components otelcol otelcol.processor.k8sattributes

otelcol.processor.k8sattributes

otelcol.processor.k8sattributes accepts telemetry data from other otelcol components and adds Kubernetes metadata to the resource attributes of spans, logs, or metrics.

You can specify multiple otelcol.processor.k8sattributes components by giving them different labels.

Usage

Alloy Copy

otelcol.processor.k8sattributes "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

The following arguments are supported:

The supported values for auth_type are:

• none: No authentication is required.
• serviceAccount: Use the built-in service account that Kubernetes automatically provisions for each pod.
• kubeConfig: Use local credentials like those used by kubectl.
• tls: Use client TLS authentication.

Setting passthrough to true enables the “passthrough mode” of otelcol.processor.k8sattributes:

• Only a k8s.pod.ip resource attribute will be added.
• No other metadata will be added.
• The Kubernetes API will not be accessed.
• To correctly detect the pod IPs, Alloy must receive spans directly from services.
• The passthrough setting is useful when configuring Alloy as a Kubernetes Deployment. A Alloy running as a Deployment cannot detect the IP addresses of pods generating telemetry data without any of the well-known IP attributes. If the Deployment Alloy receives telemetry from Alloys deployed as DaemonSet, then some of those attributes might be missing. As a workaround, you can configure the DaemonSet Alloys with passthrough set to true.

By default, otelcol.processor.k8sattributes is ready as soon as it starts, even if no metadata has been fetched yet. If telemetry is sent to this processor before the metadata is synced, there will be no metadata to enrich the telemetry with.

To wait for the metadata to be synced before otelcol.processor.k8sattributes is ready, set the wait_for_metadata option to true. Then, the processor will not be ready until the metadata is fully synced. As a result, the start-up of Alloy will be blocked. If the metadata cannot be synced by the time the metadata_sync_timeout duration is reached, otelcol.processor.k8sattributes will become unhealthy and fail to start.

If otelcol.processor.k8sattributes is unhealthy, other Alloy components will still be able to start. However, they may be unable to send telemetry to otelcol.processor.k8sattributes.

Blocks

The following blocks are supported inside the definition of otelcol.processor.k8sattributes:

The > symbol indicates deeper levels of nesting. For example, extract > annotation refers to an annotation block defined inside an extract block.

extract block

The extract block configures which metadata, annotations, and labels to extract from the pod.

The following attributes are supported:

The currently supported metadata keys are:

• k8s.pod.name
• k8s.pod.uid
• k8s.deployment.name
• k8s.node.name
• k8s.namespace.name
• k8s.pod.start_time
• k8s.replicaset.name
• k8s.replicaset.uid
• k8s.daemonset.name
• k8s.daemonset.uid
• k8s.job.name
• k8s.job.uid
• k8s.cronjob.name
• k8s.statefulset.name
• k8s.statefulset.uid
• k8s.container.name
• container.image.name
• container.image.tag
• container.id

By default, if metadata is not specified, the following fields are extracted and added to spans, metrics, and logs as resource attributes:

• k8s.pod.name
• k8s.pod.uid
• k8s.pod.start_time
• k8s.namespace.name
• k8s.node.name
• k8s.deployment.name (if the pod is controlled by a deployment)
• k8s.container.name (requires an additional attribute to be set: container.id)
• container.image.name (requires one of the following additional attributes to be set: container.id or k8s.container.name)
• container.image.tag (requires one of the following additional attributes to be set: container.id or k8s.container.name)

annotation block

The annotation block configures how to extract Kubernetes annotations.

The following attributes are supported:

Expand table

Name	Type	Description	Default	Required
from	string	The source of the labels or annotations. Allowed values are pod, namespace, and node.	pod	no
key_regex	string	A regular expression used to extract a key that matches the regular expression.	""	no
key	string	The annotation or label name. This key must exactly match an annotation or label name.	""	no
regex	string	An optional field used to extract a sub-string from a complex field value.	""	no
tag_name	string	The name of the resource attribute added to logs, metrics, or spans.	""	no

When you don’t specify the tag_name, a default tag name is used with the format:

• k8s.pod.annotations.<annotation key>
• k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your Pod spec contains the following labels:

• app.kubernetes.io/component: mysql
• app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

Alloy Copy

extract {
  label {
    from = "pod"
    key_regex = "kubernetes.io/(.*)"
    tag_name  = "$1"
  }
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to either "pod" or "namespace".

label block

The label block configures how to extract Kubernetes labels.

The following attributes are supported:

Expand table

Name	Type	Description	Default	Required
from	string	The source of the labels or annotations. Allowed values are pod, namespace, and node.	pod	no
key_regex	string	A regular expression used to extract a key that matches the regular expression.	""	no
key	string	The annotation or label name. This key must exactly match an annotation or label name.	""	no
regex	string	An optional field used to extract a sub-string from a complex field value.	""	no
tag_name	string	The name of the resource attribute added to logs, metrics, or spans.	""	no

When you don’t specify the tag_name, a default tag name is used with the format:

• k8s.pod.annotations.<annotation key>
• k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your Pod spec contains the following labels:

• app.kubernetes.io/component: mysql
• app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

Alloy Copy

extract {
  label {
    from = "pod"
    key_regex = "kubernetes.io/(.*)"
    tag_name  = "$1"
  }
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to either "pod" or "namespace".

filter block

The filter block configures which nodes to get data from and which fields and labels to fetch.

The following attributes are supported:

If node is specified, then any pods not running on the specified node will be ignored by otelcol.processor.k8sattributes.

field block

The field block allows you to filter pods by generic Kubernetes fields.

label block

The label block allows you to filter pods by generic Kubernetes labels.

pod_association block

The pod_association block configures rules on how to associate logs/traces/metrics to pods.

The pod_association block does not support any arguments and is configured fully through child blocks.

The pod_association block can be repeated multiple times, to configure additional rules.

Example:

Alloy Copy

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.ip"
    }
}

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.uid"
    }
    source {
        from = "connection"
    }
}

source block

The source block configures a pod association rule. This is used by the k8sattributes processor to determine the pod associated with a telemetry signal.

When multiple source blocks are specified inside a pod_association block, both source blocks has to match for the pod to be associated with the telemetry signal.

The following attributes are supported:

exclude block

The exclude block configures which pods to exclude from the processor.

pod block

The pod block configures a pod to be excluded from the processor.

The following attributes are supported:

output block

debug_metrics block

Exported fields

The following fields are exported and can be referenced by other components:

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.k8sattributes is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.k8sattributes does not expose any component-specific debug information.

Examples

Basic usage

In most cases, this is enough to get started. It’ll add these resource attributes to all logs, metrics, and traces:

• k8s.namespace.name
• k8s.pod.name
• k8s.pod.uid
• k8s.pod.start_time
• k8s.deployment.name
• k8s.node.name

Example:

Alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = sys.env("OTLP_ENDPOINT")
  }
}

Add additional metadata and labels

Alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from      = "pod"
      key_regex = "(.*)/(.*)"
      tag_name  = "$1.$2"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.deployment.name",
      "k8s.statefulset.name",
      "k8s.daemonset.name",
      "k8s.cronjob.name",
      "k8s.job.name",
      "k8s.node.name",
      "k8s.pod.name",
      "k8s.pod.uid",
      "k8s.pod.start_time",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = sys.env("OTLP_ENDPOINT")
  }
}

Adding Kubernetes metadata to Prometheus metrics

otelcol.processor.k8sattributes adds metadata to metrics signals in the form of resource attributes. To display the metadata as labels of Prometheus metrics, the OTLP attributes must be converted from resource attributes to datapoint attributes. One way to do this is by using an otelcol.processor.transform component.

Alloy Copy

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from = "pod"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.pod.name",
    ]
  }

  output {
    metrics = [otelcol.processor.transform.add_kube_attrs.input]
  }
}

otelcol.processor.transform "add_kube_attrs" {
  error_mode = "ignore"

  metric_statements {
    context = "datapoint"
    statements = [
      "set(attributes[\"k8s.pod.name\"], resource.attributes[\"k8s.pod.name\"])",
      "set(attributes[\"k8s.namespace.name\"], resource.attributes[\"k8s.namespace.name\"])",
    ]
  }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Compatible components

otelcol.processor.k8sattributes can accept arguments from the following components:

• Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.k8sattributes has exports that can be consumed by the following components:

• Components that consume OpenTelemetry otelcol.Consumer