Components reference on Grafana Labs

discovery.azure

Fri, 10 Apr 2026 07:58:08 +0000

discovery.azure

discovery.azure discovers Azure Virtual Machines and exposes them as targets.

Usage

discovery.azure "LABEL" {
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`environment`	`string`	Azure environment.	`"AzurePublicCloud"`	no
`port`	`number`	Port to be appended to the `__address__` label for each target.	`80`	no
`subscription_id`	`string`	Azure subscription ID.		no
`refresh_interval`	`duration`	Interval at which to refresh the list of targets.	`5m`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_from_environment uses the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof). Requests use the proxy from the environment variable matching their scheme, unless excluded by NO_PROXY. proxy_url and no_proxy must not be configured if proxy_from_environment is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.azure:

Hierarchy	Block	Description	Required
oauth	oauth	OAuth configuration for Azure API.	no
managed_identity	managed_identity	Managed Identity configuration for Azure API.	no
tls_config	tls_config	TLS configuration for requests to the Azure API.	no

Exactly one of the oauth or managed_identity blocks must be specified.

oauth block

The oauth block configures OAuth authentication for the Azure API.

Name	Type	Description	Required
`client_id`	`string`	OAuth client ID.	yes
`client_secret`	`string`	OAuth client secret.	yes
`tenant_id`	`string`	OAuth tenant ID.	yes

managed_identity block

The managed_identity block configures Managed Identity authentication for the Azure API.

Name	Type	Description	Default	Required
`client_id`	`string`	Managed Identity client ID.		yes

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

When min_version is not provided, the minimum acceptable TLS version is inherited from Go’s default minimum version, TLS 1.2. If min_version is provided, it must be set to one of the following strings:

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Azure API.

Each target includes the following labels:

__meta_azure_subscription_id: The Azure subscription ID.
__meta_azure_tenant_id: The Azure tenant ID.
__meta_azure_machine_id: The UUID of the Azure VM.
__meta_azure_machine_resource_group: The name of the resource group the VM is in.
__meta_azure_machine_name: The name of the VM.
__meta_azure_machine_computer_name: The host OS name of the VM.
__meta_azure_machine_os_type: The OS the VM is running (either Linux or Windows).
__meta_azure_machine_location: The region the VM is in.
__meta_azure_machine_private_ip: The private IP address of the VM.
__meta_azure_machine_public_ip: The public IP address of the VM.
__meta_azure_machine_tag_*: A tag on the VM. There will be one label per tag.
__meta_azure_machine_scale_set: The name of the scale set the VM is in.
__meta_azure_machine_size: The size of the VM.

Each discovered VM maps to a single target. The __address__ label is set to the private_ip:port ([private_ip]:port if the private IP is an IPv6 address) of the VM.

Component health

discovery.azure is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.azure does not expose any component-specific debug information.

Debug metrics

discovery.azure does not expose any component-specific debug metrics.

Example

discovery.azure "example" {
  port = 80
  subscription_id = AZURE_SUBSCRIPTION_ID
  oauth {
      client_id = AZURE_CLIENT_ID
      client_secret = AZURE_CLIENT_SECRET
      tenant_id = AZURE_TENANT_ID
  }
}

prometheus.scrape "demo" {
  targets    = discovery.azure.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

AZURE_SUBSCRIPTION_ID: Your Azure subscription ID.
AZURE_CLIENT_ID: Your Azure client ID.
AZURE_CLIENT_SECRET: Your Azure client secret.
AZURE_TENANT_ID: Your Azure tenant ID.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.azure has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.consul

Fri, 10 Apr 2026 07:58:08 +0000

discovery.consul

discovery.consul allows retrieving scrape targets from Consul’s Catalog API.

Usage

discovery.consul "LABEL" {
  server = CONSUL_SERVER
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	Host and port of the Consul API.	`localhost:8500`	no
`token`	`secret`	Secret token used to access the Consul API.		no
`datacenter`	`string`	Datacenter to query. If not provided, the default is used.		no
`namespace`	`string`	Namespace to use (only supported in Consul Enterprise).		no
`partition`	`string`	Admin partition to use (only supported in Consul Enterprise).		no
`tag_separator`	`string`	The string by which Consul tags are joined into the tag label.	`,`	no
`scheme`	`string`	The scheme to use when talking to Consul.	`http`	no
`username`	`string`	The username to use (deprecated in favor of the basic_auth configuration).		no
`password`	`secret`	The password to use (deprecated in favor of the basic_auth configuration).		no
`allow_stale`	`bool`	Allow stale Consul results (see official documentation). Will reduce load on Consul.	`true`	no
`services`	`list(string)`	A list of services for which targets are retrieved. If omitted, all services are scraped.		no
`tags`	`list(string)`	An optional list of tags used to filter nodes for a given service. Services must contain all tags in the list.		no
`node_meta`	`map(string)`	Node metadata key/value pairs to filter nodes for a given service.		no
`refresh_interval`	`duration`	Frequency to refresh list of containers.	`"30s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.consul:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Consul catalog API.

Each target includes the following labels:

__meta_consul_address: the address of the target.
__meta_consul_dc: the datacenter name for the target.
__meta_consul_health: the health status of the service.
__meta_consul_partition: the admin partition name where the service is registered.
__meta_consul_metadata_<key>: each node metadata key value of the target.
__meta_consul_node: the node name defined for the target.
__meta_consul_service_address: the service address of the target.
__meta_consul_service_id: the service ID of the target.
__meta_consul_service_metadata_<key>: each service metadata key value of the target.
__meta_consul_service_port: the service port of the target.
__meta_consul_service: the name of the service the target belongs to.
__meta_consul_tagged_address_<key>: each node tagged address key value of the target.
__meta_consul_tags: the list of tags of the target joined by the tag separator.

Component health

discovery.consul is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.consul does not expose any component-specific debug information.

Debug metrics

discovery.consul does not expose any component-specific debug metrics.

Example

This example discovers targets from Consul for the specified list of services:

discovery.consul "example" {
  server = "localhost:8500"
  services = [
    "service1",
    "service2",
  ]
}

prometheus.scrape "demo" {
  targets    = discovery.consul.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.consul has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.consulagent

Fri, 10 Apr 2026 07:58:08 +0000

discovery.consulagent

discovery.consulagent allows you to retrieve scrape targets from Consul’s Agent API. Only the services registered with the local agent running on the same host will be watched. This is suitable for very large Consul clusters for which using the Catalog API would be too slow or resource intensive.

Usage

discovery.consulagent "LABEL" {
  server = CONSUL_SERVER
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	Host and port of the Consul Agent API.	`localhost:8500`	no
`token`	`secret`	Secret token used to access the Consul Agent API.		no
`datacenter`	`string`	Datacenter in which the Consul Agent is configured to run. If not provided, the datacenter will be retrieved from the local Consul Agent.		no
`tag_separator`	`string`	The string by which Consul tags are joined into the tag label.	`,`	no
`scheme`	`string`	The scheme to use when talking to the Consul Agent.	`http`	no
`username`	`string`	The username to use.		no
`password`	`secret`	The password to use.		no
`services`	`list(string)`	A list of services for which targets are retrieved. If omitted, all services are scraped.		no
`tags`	`list(string)`	An optional list of tags used to filter nodes for a given service. Services must contain all tags in the list.		no
`refresh_interval`	`duration`	Frequency to refresh list of containers.	`"30s"`	no

Blocks

The following blocks are supported inside the definition of discovery.consulagent:

Hierarchy	Block	Description	Required
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Consul Agent API.

Each target includes the following labels:

__meta_consulagent_address: the address of the target.
__meta_consulagent_dc: the datacenter name for the target.
__meta_consulagent_health: the health status of the service.
__meta_consulagent_metadata_<key>: each node metadata key value of the target.
__meta_consulagent_node: the node name defined for the target.
__meta_consulagent_service: the name of the service the target belongs to.
__meta_consulagent_service_address: the service address of the target.
__meta_consulagent_service_id: the service ID of the target.
__meta_consulagent_service_metadata_<key>: each service metadata key value of the target.
__meta_consulagent_service_port: the service port of the target.
__meta_consulagent_tagged_address_<key>: each node tagged address key value of the target.
__meta_consulagent_tags: the list of tags of the target joined by the tag separator.

Component health

discovery.consulagent is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.consulagent does not expose any component-specific debug information.

Debug metrics

discovery_consulagent_rpc_failures_total (Counter): The number of Consul Agent RPC call failures.
discovery_consulagent_rpc_duration_seconds (SummaryVec): The duration of a Consul Agent RPC call in seconds.

Example

This example discovers targets from a Consul Agent for the specified list of services:

discovery.consulagent "example" {
  server = "localhost:8500"
  services = [
    "service1",
    "service2",
  ]
}

prometheus.scrape "demo" {
  targets    = discovery.consul.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.consulagent has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.digitalocean

Fri, 10 Apr 2026 07:58:08 +0000

discovery.digitalocean

discovery.digitalocean discovers DigitalOcean Droplets and exposes them as targets.

Usage

discovery.digitalocean "LABEL" {
    // Use one of:
    // bearer_token      = BEARER_TOKEN
    // bearer_token_file = PATH_TO_BEARER_TOKEN_FILE
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`port`	`number`	Port to be appended to the `__address__` label for each Droplet.	`80`	no
`refresh_interval`	`duration`	Frequency to refresh list of Droplets.	`"1m"`	no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no

The DigitalOcean API uses bearer tokens for authentication, see more about it in the DigitalOcean API documentation.

Exactly one of the bearer_token and bearer_token_file arguments must be specified to authenticate against DigitalOcean.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The discovery.digitalocean component does not support any blocks, and is configured fully through arguments.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the DigitalOcean API.

Each target includes the following labels:

__meta_digitalocean_droplet_id: ID of the Droplet.
__meta_digitalocean_droplet_name: Name of the Droplet.
__meta_digitalocean_image: The image slug (unique text identifier of the image) used to create the Droplet.
__meta_digitalocean_image_name: Name of the image used to create the Droplet.
__meta_digitalocean_private_ipv4: The private IPv4 address of the Droplet.
__meta_digitalocean_public_ipv4: The public IPv4 address of the Droplet.
__meta_digitalocean_public_ipv6: The public IPv6 address of the Droplet.
__meta_digitalocean_region: The region the Droplet is running in.
__meta_digitalocean_size: The size of the Droplet.
__meta_digitalocean_status: The current status of the Droplet.
__meta_digitalocean_features: Optional properties configured for the Droplet, such as IPV6 networking, private networking, or backups.
__meta_digitalocean_tags: The tags assigned to the Droplet.
__meta_digitalocean_vpc: The ID of the VPC where the Droplet is located.

Each discovered Droplet maps to one target.

Component health

discovery.digitalocean is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.digitalocean does not expose any component-specific debug information.

Debug metrics

discovery.digitalocean does not expose any component-specific debug metrics.

Example

This would result in targets with __address__ labels like: 192.0.2.1:8080:

discovery.digitalocean "example" {
  port             = 8080
  refresh_interval = "5m"
  bearer_token     = "my-secret-bearer-token"
}

prometheus.scrape "demo" {
  targets    = discovery.digitalocean.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.digitalocean has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.dns

Fri, 10 Apr 2026 07:58:08 +0000

discovery.dns

discovery.dns discovers scrape targets from DNS records.

Usage

discovery.dns "LABEL" {
  names = [NAME_1, NAME_2, ...]
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`names`	`list(string)`	DNS names to look up.		yes
`port`	`number`	Port to use for collecting metrics. Not used for SRV records.	`0`	no
`refresh_interval`	`duration`	How often to query DNS for updates.	`"30s"`	no
`type`	`string`	Type of DNS record to query. Must be one of SRV, A, AAAA, or MX.	`"SRV"`	no

Exported fields

The following field is exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the docker API.

Each target includes the following labels:

__meta_dns_name: Name of the record that produced the discovered target.
__meta_dns_srv_record_target: Target field of the SRV record.
__meta_dns_srv_record_port: Port field of the SRV record.
__meta_dns_mx_record_target: Target field of the MX record.

Component health

discovery.dns is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.dns does not expose any component-specific debug information.

Debug metrics

discovery.dns does not expose any component-specific debug metrics.

Example

This example discovers targets from an A record.

discovery.dns "dns_lookup" {
  names = ["myservice.example.com", "myotherservice.example.com"]
  type = "A"
  port = 8080
}

prometheus.scrape "demo" {
  targets    = discovery.dns.dns_lookup.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.dns has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.docker

Fri, 10 Apr 2026 07:58:08 +0000

discovery.docker

discovery.docker discovers Docker Engine containers and exposes them as targets.

Usage

discovery.docker "LABEL" {
  host = DOCKER_ENGINE_HOST
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`host`	`string`	Address of the Docker Daemon to connect to.		yes
`port`	`number`	Port to use for collecting metrics when containers don’t have any port mappings.	`80`	no
`host_networking_host`	`string`	Host to use if the container is in host networking mode.	`"localhost"`	no
`refresh_interval`	`duration`	Frequency to refresh list of containers.	`"1m"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.docker:

Hierarchy	Block	Description	Required
filter	filter	Filters discoverable resources.	no
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

The filter block configures a filter to pass to the Docker Engine to limit the amount of containers returned. The filter block can be specified multiple times to provide more than one filter.

Name	Type	Description	Default	Required
`name`	`string`	Filter name to use.		yes
`values`	`list(string)`	Values to pass to the filter.		yes

Refer to List containers from the Docker Engine API documentation for the list of supported filters and their meaning.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the docker API.

Each target includes the following labels:

__meta_docker_container_id: ID of the container.
__meta_docker_container_name: Name of the container.
__meta_docker_container_network_mode: Network mode of the container.
__meta_docker_container_label_<labelname>: Each label from the container.
__meta_docker_network_id: ID of the Docker network the container is in.
__meta_docker_network_name: Name of the Docker network the container is in.
__meta_docker_network_ingress: Set to true if the Docker network is an ingress network.
__meta_docker_network_internal: Set to true if the Docker network is an internal network.
__meta_docker_network_label_<labelname>: Each label from the network the container is in.
__meta_docker_network_scope: The scope of the network the container is in.
__meta_docker_network_ip: The IP of the container in the network.
__meta_docker_port_private: The private port on the container.
__meta_docker_port_public: The publicly exposed port from the container, if a port mapping exists.
__meta_docker_port_public_ip: The public IP of the container, if a port mapping exists.

Each discovered container maps to one target per unique combination of networks and port mappings used by the container.

Component health

discovery.docker is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.docker does not expose any component-specific debug information.

Debug metrics

discovery.docker does not expose any component-specific debug metrics.

Examples

Linux or macOS hosts

This example discovers Docker containers when the host machine is macOS or Linux:

discovery.docker "containers" {
  host = "unix:///var/run/docker.sock"
}

prometheus.scrape "demo" {
  targets    = discovery.docker.containers.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Windows hosts

This example discovers Docker containers when the host machine is Windows:

discovery.docker "containers" {
  host = "tcp://localhost:2375"
}

prometheus.scrape "demo" {
  targets    = discovery.docker.containers.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

NOTE: This example requires the “Expose daemon on tcp://localhost:2375 without TLS” setting to be enabled in the Docker Engine settings.

Compatible components

discovery.docker has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.dockerswarm

Fri, 10 Apr 2026 07:58:08 +0000

discovery.dockerswarm

discovery.dockerswarm allows you to retrieve scrape targets from Docker Swarm.

Usage

discovery.dockerswarm "LABEL" {
  host = "DOCKER_DAEMON_HOST"
  role = "SWARM_ROLE"
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`host`	`string`	Address of the Docker daemon.		yes
`role`	`string`	Role of the targets to retrieve. Must be `services`, `tasks`, or `nodes`.		yes
`port`	`number`	The port to scrape metrics from, when `role` is nodes, and for discovered tasks and services that don’t have published ports.	`80`	no
`refresh_interval`	`duration`	Interval at which to refresh the list of targets.	`"60s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.dockerswarm:

Hierarchy	Block	Description	Required
filter	filter	Optional filter to limit the discovery process to a subset of available resources.	no
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

Filters can be used to limit the discovery process to a subset of available resources. It is possible to define multiple filter blocks within the discovery.dockerswarm block. The list of available filters depends on the role:

The following arguments can be used to configure a filter.

Name	Type	Description	Default	Required
`name`	`string`	Name of the filter.		yes
`values`	`list(string)`	List of values associated with the filter.		yes

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from Swarm.

Roles

The role attribute decides the role of the targets to retrieve.

services

The services role discovers all Swarm services and exposes their ports as targets. For each published port of a service, a single target is generated. If a service has no published ports, a target per service is created using the port attribute defined in the arguments.

Available meta labels:

__meta_dockerswarm_service_id: the ID of the service.
__meta_dockerswarm_service_name: the name of the service.
__meta_dockerswarm_service_mode: the mode of the service.
__meta_dockerswarm_service_endpoint_port_name: the name of the endpoint port, if available.
__meta_dockerswarm_service_endpoint_port_publish_mode: the publish mode of the endpoint port.
__meta_dockerswarm_service_label_<labelname>: each label of the service.
__meta_dockerswarm_service_task_container_hostname: the container hostname of the target, if available.
__meta_dockerswarm_service_task_container_image: the container image of the target.
__meta_dockerswarm_service_updating_status: the status of the service, if available.
__meta_dockerswarm_network_id: the ID of the network.
__meta_dockerswarm_network_name: the name of the network.
__meta_dockerswarm_network_ingress: whether the network is ingress.
__meta_dockerswarm_network_internal: whether the network is internal.
__meta_dockerswarm_network_label_<labelname>: each label of the network.
__meta_dockerswarm_network_scope: the scope of the network.

tasks

The tasks role discovers all Swarm tasks and exposes their ports as targets. For each published port of a task, a single target is generated. If a task has no published ports, a target per task is created using the port attribute defined in the arguments.

Available meta labels:

__meta_dockerswarm_container_label_<labelname>: each label of the container.
__meta_dockerswarm_task_id: the ID of the task.
__meta_dockerswarm_task_container_id: the container ID of the task.
__meta_dockerswarm_task_desired_state: the desired state of the task.
__meta_dockerswarm_task_slot: the slot of the task.
__meta_dockerswarm_task_state: the state of the task.
__meta_dockerswarm_task_port_publish_mode: the publish mode of the task port.
__meta_dockerswarm_service_id: the ID of the service.
__meta_dockerswarm_service_name: the name of the service.
__meta_dockerswarm_service_mode: the mode of the service.
__meta_dockerswarm_service_label_<labelname>: each label of the service.
__meta_dockerswarm_network_id: the ID of the network.
__meta_dockerswarm_network_name: the name of the network.
__meta_dockerswarm_network_ingress: whether the network is ingress.
__meta_dockerswarm_network_internal: whether the network is internal.
__meta_dockerswarm_network_label_<labelname>: each label of the network.
__meta_dockerswarm_network_label: each label of the network.
__meta_dockerswarm_network_scope: the scope of the network.
__meta_dockerswarm_node_id: the ID of the node.
__meta_dockerswarm_node_hostname: the hostname of the node.
__meta_dockerswarm_node_address: the address of the node.
__meta_dockerswarm_node_availability: the availability of the node.
__meta_dockerswarm_node_label_<labelname>: each label of the node.
__meta_dockerswarm_node_platform_architecture: the architecture of the node.
__meta_dockerswarm_node_platform_os: the operating system of the node.
__meta_dockerswarm_node_role: the role of the node.
__meta_dockerswarm_node_status: the status of the node.

The __meta_dockerswarm_network_* meta labels are not populated for ports which are published with mode=host.

nodes

The nodes role is used to discover Swarm nodes.

Available meta labels:

__meta_dockerswarm_node_address: the address of the node.
__meta_dockerswarm_node_availability: the availability of the node.
__meta_dockerswarm_node_engine_version: the version of the node engine.
__meta_dockerswarm_node_hostname: the hostname of the node.
__meta_dockerswarm_node_id: the ID of the node.
__meta_dockerswarm_node_label_<labelname>: each label of the node.
__meta_dockerswarm_node_manager_address: the address of the manager component of the node.
__meta_dockerswarm_node_manager_leader: the leadership status of the manager component of the node (true or false).
__meta_dockerswarm_node_manager_reachability: the reachability of the manager component of the node.
__meta_dockerswarm_node_platform_architecture: the architecture of the node.
__meta_dockerswarm_node_platform_os: the operating system of the node.
__meta_dockerswarm_node_role: the role of the node.
__meta_dockerswarm_node_status: the status of the node.

Component health

discovery.dockerswarm is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.dockerswarm does not expose any component-specific debug information.

Debug metrics

discovery.dockerswarm does not expose any component-specific debug metrics.

Example

This example discovers targets from Docker Swarm tasks:

discovery.dockerswarm "example" {
  host = "unix:///var/run/docker.sock"
  role = "tasks"

  filter {
    name = "id"
    values = ["0kzzo1i0y4jz6027t0k7aezc7"]
  }

  filter {
    name = "desired-state"
    values = ["running", "accepted"]
  }
}

prometheus.scrape "demo" {
  targets    = discovery.dockerswarm.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.dockerswarm has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.ec2

Fri, 10 Apr 2026 07:58:08 +0000

discovery.ec2

discovery.ec2 lets you retrieve scrape targets from EC2 instances. The private IP address is used by default, but you can change it to the public IP address using relabeling.

The IAM credentials used must have the ec2:DescribeInstances permission to discover scrape targets, and may optionally have the ec2:DescribeAvailabilityZones permission to make the availability zone ID available as a label.

Usage

discovery.ec2 "LABEL" {
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	Custom endpoint to be used.		no
`region`	`string`	The AWS region. If blank, the region from the instance metadata is used.		no
`access_key`	`string`	The AWS API key ID. If blank, the environment variable `AWS_ACCESS_KEY_ID` is used.		no
`secret_key`	`string`	The AWS API key secret. If blank, the environment variable `AWS_SECRET_ACCESS_KEY` is used.		no
`profile`	`string`	Named AWS profile used to connect to the API.		no
`role_arn`	`string`	AWS Role Amazon Resource Name (ARN), an alternative to using AWS API keys.		no
`refresh_interval`	`string`	Refresh interval to re-read the instance list.	60s	no
`port`	`int`	The port to scrape metrics from. If using the public IP address, this must instead be specified in the relabeling rule.	80	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.ec2:

Hierarchy	Block	Description	Required
basic_auth	[basic_auth][]	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
filter	filter	Filters discoverable resources.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

Filters can be used optionally to filter the instance list by other criteria. Available filter criteria can be found in the Amazon EC2 documentation.

Name	Type	Description	Default	Required
`name`	`string`	Filter name to use.		yes
`values`	`list(string)`	Values to pass to the filter.		yes

Refer to the Filter API AWS EC2 documentation for the list of supported filters and their descriptions.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of discovered EC2 targets.

Each target includes the following labels:

__meta_ec2_ami: The EC2 Amazon Machine Image.
__meta_ec2_architecture: The architecture of the instance.
__meta_ec2_availability_zone: The availability zone in which the instance is running.
__meta_ec2_availability_zone_id: The availability zone ID in which the instance is running (requires ec2:DescribeAvailabilityZones).
__meta_ec2_instance_id: The EC2 instance ID.
__meta_ec2_instance_lifecycle: The lifecycle of the EC2 instance, set only for ‘spot’ or ‘scheduled’ instances, absent otherwise.
__meta_ec2_instance_state: The state of the EC2 instance.
__meta_ec2_instance_type: The type of the EC2 instance.
__meta_ec2_ipv6_addresses: Comma-separated list of IPv6 addresses assigned to the instance’s network interfaces, if present.
__meta_ec2_owner_id: The ID of the AWS account that owns the EC2 instance.
__meta_ec2_platform: The Operating System platform, set to ‘windows’ on Windows servers, absent otherwise.
__meta_ec2_primary_subnet_id: The subnet ID of the primary network interface, if available.
__meta_ec2_private_dns_name: The private DNS name of the instance, if available.
__meta_ec2_private_ip: The private IP address of the instance, if present.
__meta_ec2_public_dns_name: The public DNS name of the instance, if available.
__meta_ec2_public_ip: The public IP address of the instance, if available.
__meta_ec2_region: The region of the instance.
__meta_ec2_subnet_id: Comma-separated list of subnets IDs in which the instance is running, if available.
__meta_ec2_tag_<tagkey>: Each tag value of the instance.
__meta_ec2_vpc_id: The ID of the VPC in which the instance is running, if available.

Component health

discovery.ec2 is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.ec2 does not expose any component-specific debug information.

Debug metrics

discovery.ec2 does not expose any component-specific debug metrics.

Example

discovery.ec2 "ec2" {
  region = "us-east-1"
}

prometheus.scrape "demo" {
  targets    = discovery.ec2.ec2.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.ec2 has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.eureka

Fri, 10 Apr 2026 07:58:08 +0000

discovery.eureka

discovery.eureka discovers instances in a Eureka Registry and exposes them as targets.

Usage

discovery.eureka "LABEL" {
    server = SERVER
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	Eureka server URL.		yes
`refresh_interval`	`duration`	Interval at which to refresh the list of targets.	`30s`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.eureka:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Eureka API.

Each target includes the following labels:

__meta_eureka_app_name
__meta_eureka_app_instance_hostname
__meta_eureka_app_instance_homepage_url
__meta_eureka_app_instance_statuspage_url
__meta_eureka_app_instance_healthcheck_url
__meta_eureka_app_instance_ip_addr
__meta_eureka_app_instance_vip_address
__meta_eureka_app_instance_secure_vip_address
__meta_eureka_app_instance_status
__meta_eureka_app_instance_port
__meta_eureka_app_instance_port_enabled
__meta_eureka_app_instance_secure_port
__meta_eureka_app_instance_secure_port_enabled
__meta_eureka_app_instance_datacenterinfo_name
__meta_eureka_app_instance_datacenterinfo_metadata_
__meta_eureka_app_instance_country_id
__meta_eureka_app_instance_id
__meta_eureka_app_instance_metadata_

Component health

discovery.eureka is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.eureka does not expose any component-specific debug information.

Debug metrics

discovery.eureka does not expose any component-specific debug metrics.

Example

discovery.eureka "example" {
    server = "https://eureka.example.com/eureka/v1"
}

prometheus.scrape "demo" {
  targets    = discovery.eureka.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.eureka has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.file

Fri, 10 Apr 2026 07:58:08 +0000

discovery.file

NOTE: In Grafana Agent v0.35.0, the discovery.file component was renamed to local.file_match, and discovery.file was repurposed to discover scrape targets from one or more files.

If you are trying to discover files on the local filesystem rather than scrape targets within a set of files, you should use local.file_match instead.

discovery.file discovers targets from a set of files, similar to the Prometheus file_sd_config.

Usage

discovery.file "LABEL" {
  files = [FILE_PATH_1, FILE_PATH_2, ...]
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`files`	`list(string)`	Files to read and discover targets from.		yes
`refresh_interval`	`duration`	How often to sync targets.	“5m”	no

The last path segment of each element in files may contain a single * that matches any character sequence, e.g. my/path/tg_*.json.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the filesystem.

Each target includes the following labels:

__meta_filepath: The absolute path to the file the target was discovered from.

Component health

discovery.file is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.file does not expose any component-specific debug information.

Debug metrics

discovery.file does not expose any component-specific debug metrics.

Examples

Example target files

[
  {
    "targets": [ "127.0.0.1:9091", "127.0.0.1:9092" ],
    "labels": {
      "environment": "dev"
    }
  },
  {
    "targets": [ "127.0.0.1:9093" ],
    "labels": {
      "environment": "prod"
    }
  }
]

- targets:
  - 127.0.0.1:9999
  - 127.0.0.1:10101
  labels:
    job: worker
- targets:
  - 127.0.0.1:9090
  labels:
    job: prometheus

Basic file discovery

This example discovers targets from a single file, scrapes them, and writes metrics to a Prometheus remote write endpoint.

discovery.file "example" {
  files = ["/tmp/example.json"]
}

prometheus.scrape "default" {
  targets    = discovery.file.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

File discovery with retained file path label

This example discovers targets from a wildcard file path, scrapes them, and writes metrics to a Prometheus remote write endpoint.

It also uses a relabeling rule to retain the file path as a label on each target.

discovery.file "example" {
  files = ["/tmp/example_*.yaml"]
}

discovery.relabel "keep_filepath" {
  targets = discovery.file.example.targets
  rule {
    source_labels = ["__meta_filepath"]
    target_label = "filepath"
  }
}

prometheus.scrape "default" {
  targets    = discovery.relabel.keep_filepath.output
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.file has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.gce

Fri, 10 Apr 2026 07:58:08 +0000

discovery.gce

discovery.gce allows retrieving scrape targets from Google Compute Engine (GCE) instances. The private IP address is used by default, but may be changed to the public IP address with relabeling.

Credentials are discovered by the Google Cloud SDK default client by looking in the following places, preferring the first location found:

a JSON file specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable.
a JSON file in the well-known path $HOME/.config/gcloud/application_default_credentials.json.
fetched from the GCE metadata server.

If the Agent is running within GCE, the service account associated with the instance it is running on should have at least read-only permissions to the compute resources. If running outside of GCE make sure to create an appropriate service account and place the credential file in one of the expected locations.

Usage

discovery.gce "LABEL" {
  project = PROJECT_NAME
  zone    = ZONE_NAME
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`project`	`string`	The GCP Project.		yes
`zone`	`string`	The zone of the scrape targets.		yes
`filter`	`string`	Filter can be used optionally to filter the instance list by other criteria.		no
`refresh_interval`	`duration`	Refresh interval to re-read the instance list.	`"60s"`	no
`port`	`int`	The port to scrape metrics from. If using the public IP address, this must instead be specified in the relabeling rule.	`80`	no
`tag_separator`	`string`	The tag separator is used to separate the tags on concatenation.	`","`	no

For more information on the syntax of the filter argument, refer to Google’s filter documentation for Method: instances.list.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of discovered GCE targets.

Each target includes the following labels:

__meta_gce_instance_id: the numeric id of the instance
__meta_gce_instance_name: the name of the instance
__meta_gce_label_LABEL_NAME: each GCE label of the instance
__meta_gce_machine_type: full or partial URL of the machine type of the instance
__meta_gce_metadata_NAME: each metadata item of the instance
__meta_gce_network: the network URL of the instance
__meta_gce_private_ip: the private IP address of the instance
__meta_gce_interface_ipv4_NAME: IPv4 address of each named interface
__meta_gce_project: the GCP project in which the instance is running
__meta_gce_public_ip: the public IP address of the instance, if present
__meta_gce_subnetwork: the subnetwork URL of the instance
__meta_gce_tags: comma separated list of instance tags
__meta_gce_zone: the GCE zone URL in which the instance is running

Component health

discovery.gce is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.gce does not expose any component-specific debug information.

Debug metrics

discovery.gce does not expose any component-specific debug metrics.

Example

discovery.gce "gce" {
  project = "agent"
  zone    = "us-east1-a"
}

prometheus.scrape "demo" {
  targets    = discovery.gce.gce.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.gce has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.hetzner

Fri, 10 Apr 2026 07:58:08 +0000

discovery.hetzner

discovery.hetzner allows retrieving scrape targets from Hetzner Cloud API and Robot API. This service discovery uses the public IPv4 address by default, but that can be changed with relabeling.

Usage

discovery.hetzner "LABEL" {
  role = HETZNER_ROLE
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`role`	`string`	Hetzner role of entities that should be discovered.		yes
`port`	`int`	The port to scrape metrics from.	`80`	no
`refresh_interval`	`duration`	The time after which the servers are refreshed.	`"60s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

role must be one of robot or hcloud.

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.hetzner:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Hetzner catalog API.

Each target includes the following labels:

__meta_hetzner_server_id: the ID of the server
__meta_hetzner_server_name: the name of the server
__meta_hetzner_server_status: the status of the server
__meta_hetzner_public_ipv4: the public ipv4 address of the server
__meta_hetzner_public_ipv6_network: the public ipv6 network (/64) of the server
__meta_hetzner_datacenter: the datacenter of the server

`hcloud`

The labels below are only available for targets with role set to hcloud:

__meta_hetzner_hcloud_image_name: the image name of the server
__meta_hetzner_hcloud_image_description: the description of the server image
__meta_hetzner_hcloud_image_os_flavor: the OS flavor of the server image
__meta_hetzner_hcloud_image_os_version: the OS version of the server image
__meta_hetzner_hcloud_datacenter_location: the location of the server
__meta_hetzner_hcloud_datacenter_location_network_zone: the network zone of the server
__meta_hetzner_hcloud_server_type: the type of the server
__meta_hetzner_hcloud_cpu_cores: the CPU cores count of the server
__meta_hetzner_hcloud_cpu_type: the CPU type of the server (shared or dedicated)
__meta_hetzner_hcloud_memory_size_gb: the amount of memory of the server (in GB)
__meta_hetzner_hcloud_disk_size_gb: the disk size of the server (in GB)
__meta_hetzner_hcloud_private_ipv4_<networkname>: the private ipv4 address of the server within a given network
__meta_hetzner_hcloud_label_<labelname>: each label of the server
__meta_hetzner_hcloud_labelpresent_<labelname>: true for each label of the server

`robot`

The labels below are only available for targets with role set to robot:

__meta_hetzner_robot_product: the product of the server
__meta_hetzner_robot_cancelled: the server cancellation status

Component health

discovery.hetzner is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.hetzner does not expose any component-specific debug information.

Debug metrics

discovery.hetzner does not expose any component-specific debug metrics.

Example

This example discovers targets from Hetzner:

discovery.hetzner "example" {
  role = HETZNER_ROLE
}

prometheus.scrape "demo" {
  targets    = discovery.hetzner.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

HETZNER_ROLE: The role of the entities that should be discovered.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.hetzner has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.http

Fri, 10 Apr 2026 07:58:08 +0000

discovery.http

discovery.http provides a flexible way to define targets by querying an external http endpoint.

It fetches targets from an HTTP endpoint containing a list of zero or more target definitions. The target must reply with an HTTP 200 response. The HTTP header Content-Type must be application/json, and the body must be valid JSON.

Example response body:

[
  {
    "targets": [ "<host>", ... ],
    "labels": {
      "<labelname>": "<labelvalue>", ...
    }
  },
  ...
]

It is possible to use additional fields in the JSON to pass parameters to prometheus.scrape such as the metricsPath and scrape_interval.

As an example, the following will provide a target with a custom metricsPath, scrape interval, and timeout value:

[
   {
      "labels" : {
         "__metrics_path__" : "/api/prometheus",
         "__scheme__" : "https",
         "__scrape_interval__" : "60s",
         "__scrape_timeout__" : "10s",
         "service" : "custom-api-service"
      },
      "targets" : [
         "custom-api:443"
      ]
   },
]

It is also possible to append query parameters to the metrics path with the __param_<name> syntax.

For example, the following will call a metrics path of /health?target_data=prometheus:

[
   {
      "labels" : {
         "__metrics_path__" : "/health",
         "__scheme__" : "https",
         "__scrape_interval__" : "60s",
         "__scrape_timeout__" : "10s",
         "__param_target_data": "prometheus",
         "service" : "custom-api-service"
      },
      "targets" : [
         "custom-api:443"
      ]
   },
]

For more information on the potential labels you can use, see the prometheus.scrape technical details section, or the Prometheus Configuration documentation.

Usage

discovery.http "LABEL" {
  url = URL
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`url`	`string`	URL to scrape.		yes
`refresh_interval`	`duration`	How often to refresh targets.	`"60s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.http:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the filesystem.

Each target includes the following labels:

__meta_url: URL the target was obtained from.

Component health

discovery.http is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.http does not expose any component-specific debug information.

Debug metrics

prometheus_sd_http_failures_total (counter): Total number of refresh failures.

Examples

This example will query a url every 15 seconds and expose targets that it finds:

discovery.http "dynamic_targets" {
  url = "https://example.com/scrape_targets"
  refresh_interval = "15s"
}

Compatible components

discovery.http has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.ionos

Fri, 10 Apr 2026 07:58:08 +0000

discovery.ionos

discovery.ionos allows you to retrieve scrape targets from IONOS Cloud API.

Usage

discovery.ionos "LABEL" {
    datacenter_id = DATACENTER_ID
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`datacenter_id`	`string`	The unique ID of the data center.		yes
`refresh_interval`	`duration`	The time after which the servers are refreshed.	`60s`	no
`port`	`int`	The port to scrape metrics from.	80	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.ionos:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the IONOS Cloud API.

Each target includes the following labels:

__meta_ionos_server_availability_zone: the availability zone of the server.
__meta_ionos_server_boot_cdrom_id: the ID of the CD-ROM the server is booted from.
__meta_ionos_server_boot_image_id: the ID of the boot image or snapshot the server is booted from.
__meta_ionos_server_boot_volume_id: the ID of the boot volume.
__meta_ionos_server_cpu_family: the CPU family of the server to.
__meta_ionos_server_id: the ID of the server.
__meta_ionos_server_ip: comma separated list of all IPs assigned to the server.
__meta_ionos_server_lifecycle: the lifecycle state of the server resource.
__meta_ionos_server_name: the name of the server.
__meta_ionos_server_nic_ip_<nic_name>: comma separated list of IPs, grouped by the name of each NIC attached to the server.
__meta_ionos_server_servers_id: the ID of the servers the server belongs to.
__meta_ionos_server_state: the execution state of the server.
__meta_ionos_server_type: the type of the server.

Component health

discovery.ionos is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.ionos does not expose any component-specific debug information.

Debug metrics

discovery.ionos does not expose any component-specific debug metrics.

Example

discovery.ionos "example" {
    datacenter_id = "15f67991-0f51-4efc-a8ad-ef1fb31a480c"
}

prometheus.scrape "demo" {
  targets    = discovery.ionos.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.ionos has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.kubelet

Fri, 10 Apr 2026 07:58:08 +0000

discovery.kubelet

discovery.kubelet discovers Kubernetes pods running on the specified Kubelet and exposes them as scrape targets.

Usage

discovery.kubelet "LABEL" {
}

Requirements

The Kubelet must be reachable from the grafana-agent pod network.
Follow the Kubelet authorization documentation to configure authentication to the Kubelet API.

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`url`	`string`	URL of the Kubelet server.	“https://localhost:10250”	no
`refresh_interval`	`duration`	How often the Kubelet should be polled for scrape targets	`5s`	no
`namespaces`	`list(string)`	A list of namespaces to extract target pods from		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

The namespaces list limits the namespaces to discover resources in. If omitted, all namespaces are searched.

discovery.kubelet appends a /pods path to url to request the available pods. You can have additional paths in the url. For example, if url is https://kubernetes.default.svc.cluster.local:443/api/v1/nodes/cluster-node-1/proxy, then discovery.kubelet sends a request on https://kubernetes.default.svc.cluster.local:443/api/v1/nodes/cluster-node-1/proxy/pods

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.kubelet:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Kubelet API.

Each target includes the following labels:

__address__: The target address to scrape derived from the pod IP and container port.
__meta_kubernetes_namespace: The namespace of the pod object.
__meta_kubernetes_pod_name: The name of the pod object.
__meta_kubernetes_pod_ip: The pod IP of the pod object.
__meta_kubernetes_pod_label_<labelname>: Each label from the pod object.
__meta_kubernetes_pod_labelpresent_<labelname>: true for each label from the pod object.
__meta_kubernetes_pod_annotation_<annotationname>: Each annotation from the pod object.
__meta_kubernetes_pod_annotationpresent_<annotationname>: true for each annotation from the pod object.
__meta_kubernetes_pod_container_init: true if the container is an InitContainer.
__meta_kubernetes_pod_container_name: Name of the container the target address points to.
__meta_kubernetes_pod_container_id: ID of the container the target address points to. The ID is in the form <type>://<container_id>.
__meta_kubernetes_pod_container_image: The image the container is using.
__meta_kubernetes_pod_container_port_name: Name of the container port.
__meta_kubernetes_pod_container_port_number: Number of the container port.
__meta_kubernetes_pod_container_port_protocol: Protocol of the container port.
__meta_kubernetes_pod_ready: Set to true or false for the pod’s ready state.
__meta_kubernetes_pod_phase: Set to Pending, Running, Succeeded, Failed or Unknown in the lifecycle.
__meta_kubernetes_pod_node_name: The name of the node the pod is scheduled onto.
__meta_kubernetes_pod_host_ip: The current host IP of the pod object.
__meta_kubernetes_pod_uid: The UID of the pod object.
__meta_kubernetes_pod_controller_kind: Object kind of the pod controller.
__meta_kubernetes_pod_controller_name: Name of the pod controller.

Note: The Kubelet API used by this component is an internal API and therefore the data in the response returned from the API cannot be guaranteed between different versions of the Kubelet.

Component health

discovery.kubelet is reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.kubelet does not expose any component-specific debug information.

Debug metrics

discovery.kubelet does not expose any component-specific debug metrics.

Examples

Bearer token file authentication

This example uses a bearer token file to authenticate to the Kubelet API:

discovery.kubelet "k8s_pods" {
  bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
}

prometheus.scrape "demo" {
  targets    = discovery.kubelet.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Limit searched namespaces

This example limits the namespaces where pods are discovered using the namespaces argument:

discovery.kubelet "k8s_pods" {
  bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
  namespaces = ["default", "kube-system"]
}

prometheus.scrape "demo" {
  targets    = discovery.kubelet.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.kubelet has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.kubernetes

Fri, 10 Apr 2026 07:58:08 +0000

discovery.kubernetes

discovery.kubernetes allows you to find scrape targets from Kubernetes resources. It watches cluster state, and ensures targets are continually synced with what is currently running in your cluster.

If you supply no connection information, this component defaults to an in-cluster configuration. A kubeconfig file or manual connection settings can be used to override the defaults.

Usage

discovery.kubernetes "LABEL" {
  role = DISCOVERY_ROLE
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`api_server`	`string`	URL of Kubernetes API server.		no
`role`	`string`	Type of Kubernetes resource to query.		yes
`kubeconfig_file`	`string`	Path of kubeconfig file to use for connecting to Kubernetes.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

The role argument is required to specify what type of targets to discover. role must be one of node, pod, service, endpoints, endpointslice, or ingress.

node role

The node role discovers one target per cluster node with the address defaulting to the HTTP port of the Kubelet daemon. The target address defaults to the first existing address of the Kubernetes node object in the address type order of NodeInternalIP, NodeExternalIP, NodeLegacyHostIP, and NodeHostName.

The following labels are included for discovered nodes:

__meta_kubernetes_node_name: The name of the node object.
__meta_kubernetes_node_provider_id: The cloud provider’s name for the node object.
__meta_kubernetes_node_label_<labelname>: Each label from the node object.
__meta_kubernetes_node_labelpresent_<labelname>: Set to true for each label from the node object.
__meta_kubernetes_node_annotation_<annotationname>: Each annotation from the node object.
__meta_kubernetes_node_annotationpresent_<annotationname>: Set to true for each annotation from the node object.
__meta_kubernetes_node_address_<address_type>: The first address for each node address type, if it exists.

In addition, the instance label for the node will be set to the node name as retrieved from the API server.

service role

The service role discovers a target for each service port for each service. This is generally useful for externally monitoring a service. The address will be set to the Kubernetes DNS name of the service and respective service port.

The following labels are included for discovered services:

__meta_kubernetes_namespace: The namespace of the service object.
__meta_kubernetes_service_annotation_<annotationname>: Each annotation from the service object.
__meta_kubernetes_service_annotationpresent_<annotationname>: true for each annotation of the service object.
__meta_kubernetes_service_cluster_ip: The cluster IP address of the service. This does not apply to services of type ExternalName.
__meta_kubernetes_service_external_name: The DNS name of the service. This only applies to services of type ExternalName.
__meta_kubernetes_service_label_<labelname>: Each label from the service object.
__meta_kubernetes_service_labelpresent_<labelname>: true for each label of the service object.
__meta_kubernetes_service_name: The name of the service object.
__meta_kubernetes_service_port_name: Name of the service port for the target.
__meta_kubernetes_service_port_number: Number of the service port for the target.
__meta_kubernetes_service_port_protocol: Protocol of the service port for the target.
__meta_kubernetes_service_type: The type of the service.

pod role

The pod role discovers all pods and exposes their containers as targets. For each declared port of a container, a single target is generated.

If a container has no specified ports, a port-free target per container is created. These targets must have a port manually injected using a discovery.relabel component before metrics can be collected from them.

The following labels are included for discovered pods:

__meta_kubernetes_namespace: The namespace of the pod object.
__meta_kubernetes_pod_name: The name of the pod object.
__meta_kubernetes_pod_ip: The pod IP of the pod object.
__meta_kubernetes_pod_label_<labelname>: Each label from the pod object.
__meta_kubernetes_pod_labelpresent_<labelname>: true for each label from the pod object.
__meta_kubernetes_pod_annotation_<annotationname>: Each annotation from the pod object.
__meta_kubernetes_pod_annotationpresent_<annotationname>: true for each annotation from the pod object.
__meta_kubernetes_pod_container_init: true if the container is an InitContainer.
__meta_kubernetes_pod_container_name: Name of the container the target address points to.
__meta_kubernetes_pod_container_id: ID of the container the target address points to. The ID is in the form <type>://<container_id>.
__meta_kubernetes_pod_container_image: The image the container is using.
__meta_kubernetes_pod_container_port_name: Name of the container port.
__meta_kubernetes_pod_container_port_number: Number of the container port.
__meta_kubernetes_pod_container_port_protocol: Protocol of the container port.
__meta_kubernetes_pod_ready: Set to true or false for the pod’s ready state.
__meta_kubernetes_pod_phase: Set to Pending, Running, Succeeded, Failed or Unknown in the lifecycle.
__meta_kubernetes_pod_node_name: The name of the node the pod is scheduled onto.
__meta_kubernetes_pod_host_ip: The current host IP of the pod object.
__meta_kubernetes_pod_uid: The UID of the pod object.
__meta_kubernetes_pod_controller_kind: Object kind of the pod controller.
__meta_kubernetes_pod_controller_name: Name of the pod controller.

endpoints role

The endpoints role discovers targets from listed endpoints of a service. For each endpoint address one target is discovered per port. If the endpoint is backed by a pod, all container ports of a pod are discovered as targets even if they are not bound to an endpoint port.

The following labels are included for discovered endpoints:

__meta_kubernetes_namespace: The namespace of the endpoints object.
__meta_kubernetes_endpoints_name: The names of the endpoints object.
__meta_kubernetes_endpoints_label_<labelname>: Each label from the endpoints object.
__meta_kubernetes_endpoints_labelpresent_<labelname>: true for each label from the endpoints object.
The following labels are attached for all targets discovered directly from the endpoints list:
- __meta_kubernetes_endpoint_hostname: Hostname of the endpoint.
- __meta_kubernetes_endpoint_node_name: Name of the node hosting the endpoint.
- __meta_kubernetes_endpoint_ready: Set to true or false for the endpoint’s ready state.
- __meta_kubernetes_endpoint_port_name: Name of the endpoint port.
- __meta_kubernetes_endpoint_port_protocol: Protocol of the endpoint port.
- __meta_kubernetes_endpoint_address_target_kind: Kind of the endpoint address target.
- __meta_kubernetes_endpoint_address_target_name: Name of the endpoint address target.
If the endpoints belong to a service, all labels of the service role discovery are attached.
For all targets backed by a pod, all labels of the pod role discovery are attached.

endpointslice role

The endpointslice role discovers targets from existing Kubernetes endpoint slices. For each endpoint address referenced in the EndpointSlice object, one target is discovered. If the endpoint is backed by a pod, all container ports of a pod are discovered as targets even if they are not bound to an endpoint port.

The following labels are included for discovered endpoint slices:

__meta_kubernetes_namespace: The namespace of the endpoints object.
__meta_kubernetes_endpointslice_name: The name of endpoint slice object.
The following labels are attached for all targets discovered directly from the endpoint slice list:
- __meta_kubernetes_endpointslice_address_target_kind: Kind of the referenced object.
- __meta_kubernetes_endpointslice_address_target_name: Name of referenced object.
- __meta_kubernetes_endpointslice_address_type: The IP protocol family of the address of the target.
- __meta_kubernetes_endpointslice_endpoint_conditions_ready: Set to true or false for the referenced endpoint’s ready state.
- __meta_kubernetes_endpointslice_endpoint_topology_kubernetes_io_hostname: Name of the node hosting the referenced endpoint.
- __meta_kubernetes_endpointslice_endpoint_topology_present_kubernetes_io_hostname: true if the referenced object has a kubernetes.io/hostname annotation.
- __meta_kubernetes_endpointslice_port: Port of the referenced endpoint.
- __meta_kubernetes_endpointslice_port_name: Named port of the referenced endpoint.
- __meta_kubernetes_endpointslice_port_protocol: Protocol of the referenced endpoint.
If the endpoints belong to a service, all labels of the service role discovery are attached.
For all targets backed by a pod, all labels of the pod role discovery are attached.

ingress role

The ingress role discovers a target for each path of each ingress. This is generally useful for externally monitoring an ingress. The address will be set to the host specified in the Kubernetes Ingress’s spec block.

The following labels are included for discovered ingress objects:

__meta_kubernetes_namespace: The namespace of the ingress object.
__meta_kubernetes_ingress_name: The name of the ingress object.
__meta_kubernetes_ingress_label_<labelname>: Each label from the ingress object.
__meta_kubernetes_ingress_labelpresent_<labelname>: true for each label from the ingress object.
__meta_kubernetes_ingress_annotation_<annotationname>: Each annotation from the ingress object.
__meta_kubernetes_ingress_annotationpresent_<annotationname>: true for each annotation from the ingress object.
__meta_kubernetes_ingress_class_name: Class name from ingress spec, if present.
__meta_kubernetes_ingress_scheme: Protocol scheme of ingress, https if TLS config is set. Defaults to http.
__meta_kubernetes_ingress_path: Path from ingress spec. Defaults to /.

Blocks

The following blocks are supported inside the definition of discovery.kubernetes:

Hierarchy	Block	Description	Required
namespaces	namespaces	Information about which Kubernetes namespaces to search.	no
selectors	selectors	Information about which Kubernetes namespaces to search.	no
attach_metadata	attach_metadata	Optional metadata to attach to discovered targets.	no
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

namespaces block

The namespaces block limits the namespaces to discover resources in. If omitted, all namespaces are searched.

Name	Type	Description	Default	Required
`own_namespace`	`bool`	Include the namespace Grafana Agent Flow is running in.		no
`names`	`list(string)`	List of namespaces to search.		no

selectors block

The selectors block contains optional label and field selectors to limit the discovery process to a subset of resources.

Name	Type	Description	Required
`role`	`string`	Role of the selector.	yes
`label`	`string`	Label selector string.	no
`field`	`string`	Field selector string.	no

See Kubernetes’ documentation for Field selectors and Labels and selectors to learn more about the possible filters that can be used.

The endpoints role supports pod, service, and endpoints selectors. The pod role supports node selectors when configured with attach_metadata: {node: true}. Other roles only support selectors matching the role itself (e.g. node role can only contain node selectors).

Note: Using multiple discovery.kubernetes components with different selectors may result in a bigger load against the Kubernetes API.

Selectors are recommended for retrieving a small set of resources in a very large cluster. Smaller clusters are recommended to avoid selectors in favor of filtering with a discovery.relabel component instead.

attach_metadata block

The attach_metadata block allows to attach node metadata to discovered targets. Valid for roles: pod, endpoints, endpointslice.

Name	Type	Description	Default	Required
`node`	`bool`	Attach node metadata.		no

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Kubernetes API.

Component health

discovery.kubernetes is reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.kubernetes does not expose any component-specific debug information.

Debug metrics

discovery.kubernetes does not expose any component-specific debug metrics.

Examples

In-cluster discovery

This example uses in-cluster authentication to discover all pods:

discovery.kubernetes "k8s_pods" {
  role = "pod"
}

prometheus.scrape "demo" {
  targets    = discovery.kubernetes.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Kubeconfig authentication

This example uses a kubeconfig file to authenticate to the Kubernetes API:

discovery.kubernetes "k8s_pods" {
  role = "pod"
  kubeconfig_file = "/etc/k8s/kubeconfig.yaml"
}

prometheus.scrape "demo" {
  targets    = discovery.kubernetes.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Limit searched namespaces and filter by labels value

This example limits the searched namespaces and only selects pods with a specific label value attached to them:

discovery.kubernetes "k8s_pods" {
  role = "pod"

  selectors {
    role = "pod"
    label = "app.kubernetes.io/name=prometheus-node-exporter"
  }

  namespaces {
    names = ["myapp"]
  }
}

prometheus.scrape "demo" {
  targets    = discovery.kubernetes.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Limit to only pods on the same node

This example limits the search to pods on the same node as this Grafana Agent. This configuration could be useful if you are running Grafana Agent as a DaemonSet.

Note
This example assumes you have used Helm chart to deploy Grafana Agent Flow in Kubernetes and sets HOSTNAME to the Kubernetes host name. If you have a custom Kubernetes deployment, you must adapt this example to your configuration.

discovery.kubernetes "k8s_pods" {
  role = "pod"
  selectors {
    role = "pod"
    field = "spec.nodeName=" + coalesce(env("HOSTNAME"), constants.hostname)
  }
}

prometheus.scrape "demo" {
  targets    = discovery.kubernetes.k8s_pods.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.kubernetes has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.kuma

Fri, 10 Apr 2026 07:58:08 +0000

discovery.kuma

discovery.kuma discovers scrape target from the Kuma control plane.

Usage

discovery.kuma "LABEL" {
    server = SERVER
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	Address of the Kuma Control Plane’s MADS xDS server.		yes
`refresh_interval`	`duration`	The time to wait between polling update requests.	`"30s"`	no
`fetch_timeout`	`duration`	The time after which the monitoring assignments are refreshed.	`"2m"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

The following blocks are supported inside the definition of discovery.kuma:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Kuma API.

The following meta labels are available on targets and can be used by the discovery.relabel component:

__meta_kuma_mesh: the name of the proxy’s Mesh
__meta_kuma_dataplane: the name of the proxy
__meta_kuma_service: the name of the proxy’s associated Service
__meta_kuma_label_<tagname>: each tag of the proxy

Component health

discovery.kuma is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.kuma does not expose any component-specific debug information.

Debug metrics

discovery.kuma does not expose any component-specific debug metrics.

Example

discovery.kuma "example" {
    server     = "http://kuma-control-plane.kuma-system.svc:5676"
}
prometheus.scrape "demo" {
	targets    = discovery.kuma.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}
prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL
		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.kuma has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.lightsail

Fri, 10 Apr 2026 07:58:08 +0000

discovery.lightsail

discovery.lightsail allows retrieving scrape targets from Amazon Lightsail instances. The private IP address is used by default, but may be changed to the public IP address with relabeling.

Usage

discovery.lightsail "LABEL" {
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	Custom endpoint to be used.		no
`region`	`string`	The AWS region. If blank, the region from the instance metadata is used.		no
`access_key`	`string`	The AWS API key ID. If blank, the environment variable `AWS_ACCESS_KEY_ID` is used.		no
`secret_key`	`string`	The AWS API key secret. If blank, the environment variable `AWS_SECRET_ACCESS_KEY` is used.		no
`profile`	`string`	Named AWS profile used to connect to the API.		no
`role_arn`	`string`	AWS Role ARN, an alternative to using AWS API keys.		no
`refresh_interval`	`string`	Refresh interval to re-read the instance list.	60s	no
`port`	`int`	The port to scrape metrics from. If using the public IP address, this must instead be specified in the relabeling rule.	80	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.lightsail:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of discovered Lightsail targets.

Each target includes the following labels:

__meta_lightsail_availability_zone: The availability zone in which the instance is running.
__meta_lightsail_blueprint_id: The Lightsail blueprint ID.
__meta_lightsail_bundle_id: The Lightsail bundle ID.
__meta_lightsail_instance_name: The name of the Lightsail instance.
__meta_lightsail_instance_state: The state of the Lightsail instance.
__meta_lightsail_instance_support_code: The support code of the Lightsail instance.
__meta_lightsail_ipv6_addresses: Comma-separated list of IPv6 addresses assigned to the instance’s network interfaces, if present.
__meta_lightsail_private_ip: The private IP address of the instance.
__meta_lightsail_public_ip: The public IP address of the instance, if available.
__meta_lightsail_region: The region of the instance.
__meta_lightsail_tag_<tagkey>: Each tag value of the instance.

Component health

discovery.lightsail is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.lightsail does not expose any component-specific debug information.

Debug metrics

discovery.lightsail does not expose any component-specific debug metrics.

Example

discovery.lightsail "lightsail" {
  region = "us-east-1"
}

prometheus.scrape "demo" {
  targets    = discovery.lightsail.lightsail.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.lightsail has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.linode

Fri, 10 Apr 2026 07:58:08 +0000

discovery.linode

discovery.linode allows you to retrieve scrape targets from Linode’s Linode APIv4. This service discovery uses the public IPv4 address by default, but that can be changed with relabeling.

Usage

discovery.linode "LABEL" {
	bearer_token = LINODE_API_TOKEN
}

Note
The linode APIv4 Token must be created with the scopes: linodes:read_only, ips:read_only, and events:read_only.

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`refresh_interval`	`duration`	The time to wait between polling update requests.	`"60s"`	no
`port`	`int`	Port that metrics are scraped from.	`80`	no
`tag_separator`	`string`	The string by which Linode Instance tags are joined into the tag label.	`,`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.linode:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Linode API.

The following meta labels are available on targets and can be used by the discovery.relabel component:

__meta_linode_instance_id: the id of the Linode instance
__meta_linode_instance_label: the label of the Linode instance
__meta_linode_image: the slug of the Linode instance’s image
__meta_linode_private_ipv4: the private IPv4 of the Linode instance
__meta_linode_public_ipv4: the public IPv4 of the Linode instance
__meta_linode_public_ipv6: the public IPv6 of the Linode instance
__meta_linode_region: the region of the Linode instance
__meta_linode_type: the type of the Linode instance
__meta_linode_status: the status of the Linode instance
__meta_linode_tags: a list of tags of the Linode instance joined by the tag separator
__meta_linode_group: the display group a Linode instance is a member of
__meta_linode_hypervisor: the virtualization software powering the Linode instance
__meta_linode_backups: the backup service status of the Linode instance
__meta_linode_specs_disk_bytes: the amount of storage space the Linode instance has access to
__meta_linode_specs_memory_bytes: the amount of RAM the Linode instance has access to
__meta_linode_specs_vcpus: the number of VCPUS this Linode has access to
__meta_linode_specs_transfer_bytes: the amount of network transfer the Linode instance is allotted each month
__meta_linode_extra_ips: a list of all extra IPv4 addresses assigned to the Linode instance joined by the tag separator

Component health

discovery.linode is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.linode does not expose any component-specific debug information.

Debug metrics

discovery.linode does not expose any component-specific debug metrics.

Example

discovery.linode "example" {
    bearer_token = env("LINODE_TOKEN")
    port = 8876
}
prometheus.scrape "demo" {
	targets    = discovery.linode.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}
prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL
		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Using private IP address:

discovery.linode "example" {
    bearer_token = env("LINODE_TOKEN")
    port = 8876
}
discovery.relabel "private_ips" {
	targets = discovery.linode.example.targets
	rule {
    	source_labels = ["__meta_linode_private_ipv4"]
    	replacement     = "[$1]:8876"
    	target_label  = "__address__"
  	}
}
prometheus.scrape "demo" {
	targets    = discovery.relabel.private_ips.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}
prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL
		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Compatible components

discovery.linode has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.marathon

Fri, 10 Apr 2026 07:58:08 +0000

discovery.marathon

discovery.marathon allows you to retrieve scrape targets from Marathon’s Service API.

Usage

discovery.marathon "LABEL" {
  servers = [MARATHON_SERVER1, MARATHON_SERVER2...]
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`servers`	`list(string)`	List of Marathon servers.		yes
`refresh_interval`	`duration`	Interval at which to refresh the list of targets.	`"30s"`	no
`auth_token`	`secret`	Auth token to authenticate with.		no
`auth_token_file`	`string`	File containing an auth token to authenticate with.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.marathon:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Marathon servers.

Each target includes the following labels:

__meta_marathon_app: the name of the app (with slashes replaced by dashes).
__meta_marathon_image: the name of the Docker image used (if available).
__meta_marathon_task: the ID of the Mesos task.
__meta_marathon_app_label_<labelname>: any Marathon labels attached to the app.
__meta_marathon_port_definition_label_<labelname>: the port definition labels.
__meta_marathon_port_mapping_label_<labelname>: the port mapping labels.
__meta_marathon_port_index: the port index number (e.g. 1 for PORT1).

Component health

discovery.marathon is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.marathon does not expose any component-specific debug information.

Debug metrics

discovery.marathon does not expose any component-specific debug metrics.

Example

This example discovers targets from a Marathon server:

discovery.marathon "example" {
  servers = ["localhost:8500"]
}

prometheus.scrape "demo" {
  targets    = discovery.marathon.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.marathon has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.nerve

Fri, 10 Apr 2026 07:58:08 +0000

discovery.nerve

discovery.nerve discovers airbnb/nerve targets stored in Zookeeper.

Usage

discovery.nerve "LABEL" {
	servers = [SERVER_1, SERVER_2]
	paths   = [PATH_1, PATH_2]
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`servers`	`list(string)`	The Zookeeper servers.		yes
`paths`	`list(string)`	The paths to look for targets at.		yes
`timeout`	`duration`	The timeout to use.	`"10s"`	no

Each element in the path list can either point to a single service, or to the root of a tree of services.

Blocks

The discovery.nerve component does not support any blocks, and is configured fully through arguments.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from Nerve’s API.

The following meta labels are available on targets and can be used by the discovery.relabel component

__meta_nerve_path: the full path to the endpoint node in Zookeeper
__meta_nerve_endpoint_host: the host of the endpoint
__meta_nerve_endpoint_port: the port of the endpoint
__meta_nerve_endpoint_name: the name of the endpoint

Component health

discovery.nerve is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.nerve does not expose any component-specific debug information.

Debug metrics

discovery.nerve does not expose any component-specific debug metrics.

Example

discovery.nerve "example" {
	servers = ["localhost"]
	paths   = ["/monitoring"]
	timeout = "1m"
}
prometheus.scrape "demo" {
	targets    = discovery.nerve.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}
prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL
		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.nerve has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.nomad

Fri, 10 Apr 2026 07:58:08 +0000

discovery.nomad

discovery.nomad allows you to retrieve scrape targets from Nomad’s Service API.

Usage

discovery.nomad "LABEL" {
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	Address of nomad server.	`http://localhost:4646`	no
`namespace`	`string`	Nomad namespace to use.	`default`	no
`region`	`string`	Nomad region to use.	`global`	no
`allow_stale`	`bool`	Allow reading from non-leader nomad instances.	`true`	no
`tag_separator`	`string`	Seperator to join nomad tags into Prometheus labels.	`,`	no
`refresh_interval`	`duration`	Frequency to refresh list of containers.	`"30s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.nomad:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the nomad server.

Each target includes the following labels:

__meta_nomad_address: the service address of the target.
__meta_nomad_dc: the datacenter name for the target.
__meta_nomad_namespace: the namespace of the target.
__meta_nomad_node_id: the node name defined for the target.
__meta_nomad_service: the name of the service the target belongs to.
__meta_nomad_service_address: the service address of the target.
__meta_nomad_service_id: the service ID of the target.
__meta_nomad_service_port: the service port of the target.
__meta_nomad_tags: the list of tags of the target joined by the tag separator.

Component health

discovery.nomad is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.nomad does not expose any component-specific debug information.

Debug metrics

discovery.nomad does not expose any component-specific debug metrics.

Example

This example discovers targets from a Nomad server:

discovery.nomad "example" {
}

prometheus.scrape "demo" {
  targets    = discovery.nomad.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.nomad has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.openstack

Fri, 10 Apr 2026 07:58:08 +0000

discovery.openstack

discovery.openstack discovers OpenStack Nova instances and exposes them as targets.

Usage

discovery.openstack "LABEL" {
  role   = "hypervisor"
  region = "us-east-1"
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`role`	`string`	Role of the discovered targets.		yes
`region`	`string`	OpenStack region.		yes
`identity_endpoint`	`string`	Specifies the HTTP endpoint that is required to work with te Identity API of the appropriate version		no
`username`	`string`	OpenStack username for the Identity V2 and V3 APIs.		no
`userid`	`string`	OpenStack userid for the Identity V2 and V3 APIs.		no
`password`	`secret`	Password for the Identity V2 and V3 APIs.		no
`domain_name`	`string`	OpenStack domain name for the Identity V2 and V3 APIs.		no
`domain_id`	`string`	OpenStack domain ID for the Identity V2 and V3 APIs.		no
`project_name`	`string`	OpenStack project name for the Identity V2 and V3 APIs.		no
`project_id`	`string`	OpenStack project ID for the Identity V2 and V3 APIs.		no
`application_credential_name`	`string`	OpenStack application credential name for the Identity V2 and V3 APIs.		no
`application_credential_id`	`string`	OpenStack application credential ID for the Identity V2 and V3 APIs.		no
`application_credential_secret`	`secret`	OpenStack application credential secret for the Identity V2 and V3 APIs.		no
`all_tenants`	`bool`	Whether the service discovery should list all instances for all projects.	`false`	no
`refresh_interval`	`duration`	Refresh interval to re-read the instance list.	`60s`	no
`port`	`int`	The port to scrape metrics from.	`80`	no
`availability`	`string`	The availability of the endpoint to connect to.	`public`	no

role must be one of hypervisor or instance.

username is required if using Identity V2 API. In Identity V3, either userid or a combination of username and domain_id or domain_name are needed.

project_id and project_name fields are optional for the Identity V2 API. Some providers allow you to specify a project_name instead of the project_id. Some require both.

application_credential_id or application_credential_name fields are required if using an application credential to authenticate. Some providers allow you to create an application credential to authenticate rather than a password.

application_credential_secret field is required if using an application credential to authenticate.

all_tenants is only relevant for the instance role and usually requires admin permissions.

availability must be one of public, admin, or internal.

Blocks

The following blocks are supported inside the definition of discovery.openstack:

Hierarchy	Block	Description	Required
tls_config	tls_config	TLS configuration for requests to the OpenStack API.	no

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the OpenStack API.

`hypervisor`

The hypervisor role discovers one target per Nova hypervisor node. The target address defaults to the host_ip attribute of the hypervisor.

__meta_openstack_hypervisor_host_ip: the hypervisor node’s IP address.
__meta_openstack_hypervisor_hostname: the hypervisor node’s name.
__meta_openstack_hypervisor_id: the hypervisor node’s ID.
__meta_openstack_hypervisor_state: the hypervisor node’s state.
__meta_openstack_hypervisor_status: the hypervisor node’s status.
__meta_openstack_hypervisor_type: the hypervisor node’s type.

`instance`

The instance role discovers one target per network interface of Nova instance. The target address defaults to the private IP address of the network interface.

__meta_openstack_address_pool: the pool of the private IP.
__meta_openstack_instance_flavor: the flavor of the OpenStack instance.
__meta_openstack_instance_id: the OpenStack instance ID.
__meta_openstack_instance_image: the ID of the image the OpenStack instance is using.
__meta_openstack_instance_name: the OpenStack instance name.
__meta_openstack_instance_status: the status of the OpenStack instance.
__meta_openstack_private_ip: the private IP of the OpenStack instance.
__meta_openstack_project_id: the project (tenant) owning this instance.
__meta_openstack_public_ip: the public IP of the OpenStack instance.
__meta_openstack_tag_<tagkey>: each tag value of the instance.
__meta_openstack_user_id: the user account owning the tenant.

Component health

discovery.openstack is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.openstack does not expose any component-specific debug information.

Debug metrics

discovery.openstack does not expose any component-specific debug metrics.

Example

discovery.openstack "example" {
  role   = OPENSTACK_ROLE
  region = OPENSTACK_REGION
}

prometheus.scrape "demo" {
  targets    = discovery.openstack.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

OPENSTACK_ROLE: Your OpenStack role.
OPENSTACK_REGION: Your OpenStack region.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.openstack has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.ovhcloud

Fri, 10 Apr 2026 07:58:08 +0000

discovery.ovhcloud

discovery.ovhcloud discovers scrape targets from OVHcloud’s dedicated servers and VPS using their API. Grafana Agent will periodically check the REST endpoint and create a target for every discovered server. The public IPv4 address will be used by default - if there’s none, the IPv6 address will be used. This may be changed via relabeling with discovery.relabel. For OVHcloud’s public cloud instances you can use discovery.openstack.

Usage

discovery.ovhcloud "LABEL" {
    application_key    = APPLICATION_KEY
    application_secret = APPLICATION_SECRET
    consumer_key       = CONSUMER_KEY
    service            = SERVICE
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
application_key	`string`	API application key.		yes
application_secret	`secret`	API application secret.		yes
consumer_key	`secret`	API consumer key.		yes
endpoint	`string`	API endpoint.	“ovh-eu”	no
refresh_interval	`duration`	Refresh interval to re-read the resources list.	“60s”	no
service	`string`	Service of the targets to retrieve.		yes

endpoint must be one of the supported API endpoints.

service must be either vps or dedicated_server.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the OVHcloud API.

Multiple meta labels are available on targets and can be used by the discovery.relabel component.

VPS meta labels:

__meta_ovhcloud_vps_cluster: the cluster of the server.
__meta_ovhcloud_vps_datacenter: the datacenter of the server.
__meta_ovhcloud_vps_disk: the disk of the server.
__meta_ovhcloud_vps_display_name: the display name of the server.
__meta_ovhcloud_vps_ipv4: the IPv4 of the server.
__meta_ovhcloud_vps_ipv6: the IPv6 of the server.
__meta_ovhcloud_vps_keymap: the KVM keyboard layout of the server.
__meta_ovhcloud_vps_maximum_additional_ip: the maximum additional IPs of the server.
__meta_ovhcloud_vps_memory_limit: the memory limit of the server.
__meta_ovhcloud_vps_memory: the memory of the server.
__meta_ovhcloud_vps_monitoring_ip_blocks: the monitoring IP blocks of the server.
__meta_ovhcloud_vps_name: the name of the server.
__meta_ovhcloud_vps_netboot_mode: the netboot mode of the server.
__meta_ovhcloud_vps_offer_type: the offer type of the server.
__meta_ovhcloud_vps_offer: the offer of the server.
__meta_ovhcloud_vps_state: the state of the server.
__meta_ovhcloud_vps_vcore: the number of virtual cores of the server.
__meta_ovhcloud_vps_version: the version of the server.
__meta_ovhcloud_vps_zone: the zone of the server.

Dedicated servers meta labels:

__meta_ovhcloud_dedicated_server_commercial_range: the commercial range of the server.
__meta_ovhcloud_dedicated_server_datacenter: the datacenter of the server.
__meta_ovhcloud_dedicated_server_ipv4: the IPv4 of the server.
__meta_ovhcloud_dedicated_server_ipv6: the IPv6 of the server.
__meta_ovhcloud_dedicated_server_link_speed: the link speed of the server.
__meta_ovhcloud_dedicated_server_name: the name of the server.
__meta_ovhcloud_dedicated_server_os: the operating system of the server.
__meta_ovhcloud_dedicated_server_rack: the rack of the server.
__meta_ovhcloud_dedicated_server_reverse: the reverse DNS name of the server.
__meta_ovhcloud_dedicated_server_server_id: the ID of the server.
__meta_ovhcloud_dedicated_server_state: the state of the server.
__meta_ovhcloud_dedicated_server_support_level: the support level of the server.

Component health

discovery.ovhcloud is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.ovhcloud does not expose any component-specific debug information.

Debug metrics

discovery.ovhcloud does not expose any component-specific debug metrics.

Example

discovery.ovhcloud "example" {
	application_key    = APPLICATION_KEY
	application_secret = APPLICATION_SECRET
	consumer_key       = CONSUMER_KEY
	service            = SERVICE
}

prometheus.scrape "demo" {
	targets    = discovery.ovhcloud.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL
		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

APPLICATION_KEY: The OVHcloud API application key.
APPLICATION_SECRET: The OVHcloud API application secret.
CONSUMER_KEY: The OVHcloud API consumer key.
SERVICE: The OVHcloud service of the targets to retrieve.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.ovhcloud has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.process

Fri, 10 Apr 2026 07:58:08 +0000

discovery.process

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

discovery.process discovers processes running on the local Linux OS.

Note
To use the discovery.process component you must run Grafana Agent Flow as root and inside host PID namespace.

Usage

discovery.process "LABEL" {

}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`join`	`list(map(string))`	Join external targets to discovered processes targets based on `__container_id__` label.		no
`refresh_interval`	`duration`	How often to sync targets.	“60s”	no

Targets joining

If join is specified, discovery.process will join the discovered processes based on the __container_id__ label.

For example, if join is specified as follows:

[
  {
    "pod": "pod-1",
    "__container_id__": "container-1"
  },
  {
    "pod": "pod-2",
    "__container_id__": "container-2"
  }
]

And the discovered processes are:

[
  {
    "__process_pid__": "1",
    "__container_id__": "container-1"
  },
  {
    "__process_pid__": "2"
  }
]

The resulting targets are:

[
  {
    "__container_id__": "container-1",
    "__process_pid__": "1",
    "pod": "pod-1"
  },
  {
    "__process_pid__": "2"
  },
  {
    "__container_id__": "container-1",
    "pod": "pod-1"
  },
  {
    "__container_id__": "container-2",
    "pod": "pod-2"
  }
]

Blocks

The following blocks are supported inside the definition of discovery.process:

Hierarchy	Block	Description	Required
discover_config	discover_config	Configures which process metadata to discover.	no

discover_config block

The discover_config block describes which process metadata to discover.

The following arguments are supported:

Name	Type	Description	Default	Required
`exe`	`bool`	A flag to enable discovering `__meta_process_exe` label.	true	no
`cwd`	`bool`	A flag to enable discovering `__meta_process_cwd` label.	true	no
`commandline`	`bool`	A flag to enable discovering `__meta_process_commandline` label.	true	no
`uid`	`bool`	A flag to enable discovering `__meta_process_uid`: label.	true	no
`username`	`bool`	A flag to enable discovering `__meta_process_username`: label.	true	no
`container_id`	`bool`	A flag to enable discovering `__container_id__` label.	true	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of processes discovered on the local Linux OS.

Each target includes the following labels:

__process_pid__: The process PID.
__meta_process_exe: The process executable path. Taken from /proc/<pid>/exe.
__meta_process_cwd: The process current working directory. Taken from /proc/<pid>/cwd.
__meta_process_commandline: The process command line. Taken from /proc/<pid>/cmdline.
__meta_process_uid: The process UID. Taken from /proc/<pid>/status.
__meta_process_username: The process username. Taken from __meta_process_uid and os/user/LookupID.
__container_id__: The container ID. Taken from /proc/<pid>/cgroup. If the process is not running in a container, this label is not set.

Component health

discovery.process is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.process does not expose any component-specific debug information.

Debug metrics

discovery.process does not expose any component-specific debug metrics.

Examples

Example discovering processes on the local host

discovery.process "all" {
  refresh_interval = "60s"
  discover_config {
    cwd = true
    exe = true
    commandline = true
    username = true
    uid = true
    container_id = true
  }
}

Example discovering processes on the local host and joining with `discovery.kubernetes`

discovery.kubernetes "pyroscope_kubernetes" {
  selectors {
    field = "spec.nodeName=" + env("HOSTNAME")
    role = "pod"
  }
  role = "pod"
}

discovery.process "all" {
  join = discovery.kubernetes.pyroscope_kubernetes.targets
  refresh_interval = "60s"
  discover_config {
    cwd = true
    exe = true
    commandline = true
    username = true
    uid = true
    container_id = true
  }
}

Compatible components

discovery.process can accept arguments from the following components:

Components that export Targets

discovery.process has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.puppetdb

Fri, 10 Apr 2026 07:58:08 +0000

discovery.puppetdb

discovery.puppetdb allows you to retrieve scrape targets from PuppetDB resources.

This SD discovers resources and will create a target for each resource returned by the API.

The resource address is the certname of the resource, and can be changed during relabeling.

The queries for this component are expected to be valid PQL (Puppet Query Language).

Usage

discovery.puppetdb "LABEL" {
  url = PUPPET_SERVER
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`url`	`string`	The URL of the PuppetDB root query endpoint.		yes
`query`	`string`	Puppet Query Language (PQL) query. Only resources are supported.		yes
`include_parameters`	`bool`	Whether to include the parameters as meta labels. Due to the differences between parameter types and Prometheus labels, some parameters might not be rendered. The format of the parameters might also change in future releases. Make sure that you don’t have secrets exposed as parameters if you enable this.	`false`	no
`port`	`int`	The port to scrape metrics from.	`80`	no
`refresh_interval`	`duration`	Frequency to refresh targets.	`"30s"`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.puppetdb:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from puppetdb.

Each target includes the following labels:

__meta_puppetdb_query: the Puppet Query Language (PQL) query.
__meta_puppetdb_certname: the name of the node associated with the resourcet.
__meta_puppetdb_resource: a SHA-1 hash of the resource’s type, title, and parameters, for identification.
__meta_puppetdb_type: the resource type.
__meta_puppetdb_title: the resource title.
__meta_puppetdb_exported: whether the resource is exported (“true” or “false”).
__meta_puppetdb_tags: comma separated list of resource tags.
__meta_puppetdb_file: the manifest file in which the resource was declared.
__meta_puppetdb_environment: the environment of the node associated with the resource.
__meta_puppetdb_parameter_<parametername>: the parameters of the resource.

Component health

discovery.puppetdb is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.puppetdb does not expose any component-specific debug information.

Debug metrics

discovery.puppetdb does not expose any component-specific debug metrics.

Example

This example discovers targets from puppetdb for all the servers that have a specific package defined:

discovery.puppetdb "example" {
	url   = "http://puppetdb.local:8080"
	query = "resources { type = \"Package\" and title = \"node_exporter\" }"
	port  = 9100
}

prometheus.scrape "demo" {
	targets    = discovery.puppetdb.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL

		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.puppetdb has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.relabel

Fri, 10 Apr 2026 07:58:08 +0000

discovery.relabel

In Flow, targets are defined as sets of key-value pairs called labels.

discovery.relabel rewrites the label set of the input targets by applying one or more relabeling rules. If no rules are defined, then the input targets are exported as-is.

The most common use of discovery.relabel is to filter targets or standardize the target label set that is passed to a downstream component. The rule blocks are applied to the label set of each target in order of their appearance in the configuration file. The configured rules can be retrieved by calling the function in the rules export field.

Target labels which start with a double underscore __ are considered internal, and may be removed by other Flow components prior to telemetry collection. To retain any of these labels, use a labelmap action to remove the prefix, or remap them to a different name. Service discovery mechanisms usually group their labels under __meta_*. For example, the discovery.kubernetes component populates a set of __meta_kubernetes_* labels to provide information about the discovered Kubernetes resources. If a relabeling rule needs to store a label value temporarily, for example as the input to a subsequent step, use the __tmp label name prefix, as it is guaranteed to never be used.

Multiple discovery.relabel components can be specified by giving them different labels.

Usage

discovery.relabel "LABEL" {
  targets = TARGET_LIST

  rule {
    ...
  }

  ...
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`targets`	`list(map(string))`	Targets to relabel		yes

Blocks

The following blocks are supported inside the definition of discovery.relabel:

Hierarchy	Block	Description	Required
rule	rule	Relabeling rules to apply to targets.	no

rule block

The rule block contains the definition of any relabeling rules that can be applied to an input metric. If more than one rule block is defined, the transformations are applied in top-down order.

The following arguments can be used to configure a rule. All arguments are optional. Omitted fields take their default values.

Name	Type	Description	Default	Required
`action`	`string`	The relabeling action to perform.	replace	no
`modulus`	`uint`	A positive integer used to calculate the modulus of the hashed source label values.		no
`regex`	`string`	A valid RE2 expression with support for parenthesized capture groups. Used to match the extracted value from the combination of the `source_label` and `separator` fields or filter labels during the `labelkeep/labeldrop/labelmap` actions.	`(.*)`	no
`replacement`	`string`	The value against which a regular expression replace is performed, if the regular expression matches the extracted value. Supports previously captured groups.	`"$1"`	no
`separator`	`string`	The separator used to concatenate the values present in `source_labels`.	;	no
`source_labels`	`list(string)`	The list of labels whose values are to be selected. Their content is concatenated using the `separator` and matched against `regex`.		no
`target_label`	`string`	Label to which the resulting value will be written to.		no

You can use the following actions:

drop - Drops metrics where regex matches the string extracted using the source_labels and separator.
dropequal - Drop targets for which the concatenated source_labels do match target_label.
hashmod - Hashes the concatenated labels, calculates its modulo modulus and writes the result to the target_label.
keep - Keeps metrics where regex matches the string extracted using the source_labels and separator.
keepequal - Drop targets for which the concatenated source_labels do not match target_label.
labeldrop - Matches regex against all label names. Any labels that match are removed from the metric’s label set.
labelkeep - Matches regex against all label names. Any labels that don’t match are removed from the metric’s label set.
labelmap - Matches regex against all label names. Any labels that match are renamed according to the contents of the replacement field.
lowercase - Sets target_label to the lowercase form of the concatenated source_labels.
replace - Matches regex to the concatenated labels. If there’s a match, it replaces the content of the target_label using the contents of the replacement field.
uppercase - Sets target_label to the uppercase form of the concatenated source_labels.

Note
The regular expression capture groups can be referred to using either the $CAPTURE_GROUP_NUMBER or ${CAPTURE_GROUP_NUMBER} notation.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`output`	`list(map(string))`	The set of targets after applying relabeling.
`rules`	`RelabelRules`	The currently configured relabeling rules.

Component health

discovery.relabel is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.relabel does not expose any component-specific debug information.

Debug metrics

discovery.relabel does not expose any component-specific debug metrics.

Example

discovery.relabel "keep_backend_only" {
  targets = [
    { "__meta_foo" = "foo", "__address__" = "localhost", "instance" = "one",   "app" = "backend"  },
    { "__meta_bar" = "bar", "__address__" = "localhost", "instance" = "two",   "app" = "database" },
    { "__meta_baz" = "baz", "__address__" = "localhost", "instance" = "three", "app" = "frontend" },
  ]

  rule {
    source_labels = ["__address__", "instance"]
    separator     = "/"
    target_label  = "destination"
    action        = "replace"
  }

  rule {
    source_labels = ["app"]
    action        = "keep"
    regex         = "backend"
  }
}

Compatible components

discovery.relabel can accept arguments from the following components:

Components that export Targets

discovery.relabel has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.scaleway

Fri, 10 Apr 2026 07:58:08 +0000

discovery.scaleway

discovery.scaleway discovers targets from Scaleway instances and baremetal services.

Usage

discovery.scaleway "LABEL" {
    project_id = "SCALEWAY_PROJECT_ID"
    role       = "SCALEWAY_PROJECT_ROLE"
    access_key = "SCALEWAY_ACCESS_KEY"
    secret_key = "SCALEWAY_SECRET_KEY"
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`project_id`	`string`	Scaleway project ID of targets.		yes
`role`	`string`	Role of targets to retrieve.		yes
`api_url`	`string`	Scaleway API URL.	`"https://api.scaleway.com"`	no
`zone`	`string`	Availability zone of targets.	`"fr-par-1"`	no
`access_key`	`string`	Access key for the Scaleway API.		yes
`secret_key`	`secret`	Secret key for the Scaleway API.		conditional
`secret_key_file`	`string`	Path to file containing secret key for the Scaleway API.		conditional
`name_filter`	`string`	Name filter to apply against the listing request.		no
`tags_filter`	`list(string)`	List of tags to search for.		no
`refresh_interval`	`duration`	Frequency to rediscover targets.	`"60s"`	no
`port`	`number`	Default port on servers to associate with generated targets.	`80`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no

The role argument determines what type of Scaleway machines to discover. It must be set to one of the following:

"baremetal": Discover baremetal Scaleway machines.
"instance": Discover virtual Scaleway instances.

The name_filter and tags_filter arguments can be used to filter the set of discovered servers. name_filter returns machines matching a specific name, while tags_filter returns machines who contain all the tags listed in the tags_filter argument.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.scaleway:

Hierarchy	Block	Description	Required
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Consul catalog API.

When role is baremetal, discovered targets include the following labels:

__meta_scaleway_baremetal_id: ID of the server.
__meta_scaleway_baremetal_public_ipv4: Public IPv4 address of the server.
__meta_scaleway_baremetal_public_ipv6: Public IPv6 address of the server.
__meta_scaleway_baremetal_name: Name of the server.
__meta_scaleway_baremetal_os_name: Operating system name of the server.
__meta_scaleway_baremetal_os_version: Operation system version of the server.
__meta_scaleway_baremetal_project_id: Project ID the server belongs to.
__meta_scaleway_baremetal_status: Current status of the server.
__meta_scaleway_baremetal_tags: The list of tags associated with the server concatenated with a ,.
__meta_scaleway_baremetal_type: Commercial type of the server.
__meta_scaleway_baremetal_zone: Availability zone of the server.

When role is instance, discovered targets include the following labels:

__meta_scaleway_instance_boot_type: Boot type of the server.
__meta_scaleway_instance_hostname: Hostname of the server.
__meta_scaleway_instance_id: ID of the server.
__meta_scaleway_instance_image_arch: Architecture of the image the server is running.
__meta_scaleway_instance_image_id: ID of the image the server is running.
__meta_scaleway_instance_image_name: Name of the image the server is running.
__meta_scaleway_instance_location_cluster_id: ID of the cluster for the server’s location.
__meta_scaleway_instance_location_hypervisor_id: Hypervisor ID for the server’s location.
__meta_scaleway_instance_location_node_id: Node ID for the server’s location.
__meta_scaleway_instance_name: Name of the server.
__meta_scaleway_instance_organization_id: Organization ID that the server belongs to.
__meta_scaleway_instance_private_ipv4: Private IPv4 address of the server.
__meta_scaleway_instance_project_id: Project ID the server belongs to.
__meta_scaleway_instance_public_ipv4: Public IPv4 address of the server.
__meta_scaleway_instance_public_ipv6: Public IPv6 address of the server.
__meta_scaleway_instance_region: Region of the server.
__meta_scaleway_instance_security_group_id: ID of the security group the server is assigned to.
__meta_scaleway_instance_security_group_name: Name of the security group the server is assigned to.
__meta_scaleway_instance_status: Current status of the server.
__meta_scaleway_instance_tags: The list of tags associated with the server concatenated with a ,.
__meta_scaleway_instance_type: Commercial type of the server.
__meta_scaleway_instance_zone: Availability zone of the server.

Component health

discovery.scaleway is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.scaleway does not expose any component-specific debug information.

Debug metrics

discovery.scaleway does not expose any component-specific debug metrics.

Example

discovery.scaleway "example" {
    project_id = "SCALEWAY_PROJECT_ID"
    role       = "SCALEWAY_PROJECT_ROLE"
    access_key = "SCALEWAY_ACCESS_KEY"
    secret_key = "SCALEWAY_SECRET_KEY"
}

prometheus.scrape "demo" {
    targets    = discovery.scaleway.example.targets
    forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
    endpoint {
        url = PROMETHEUS_REMOTE_WRITE_URL

        basic_auth {
            username = USERNAME
            password = PASSWORD
        }
    }
}

Replace the following:

SCALEWAY_PROJECT_ID: The project ID of your Scaleway machines.
SCALEWAY_PROJECT_ROLE: Set to baremetal to discover baremetal machines or instance to discover virtual instances.
SCALEWAY_ACCESS_KEY: Your Scaleway API access key.
SCALEWAY_SECRET_KEY: Your Scaleway API secret key.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.scaleway has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.serverset

Fri, 10 Apr 2026 07:58:08 +0000

discovery.serverset

discovery.serverset discovers Serversets stored in Zookeeper and exposes them as targets. Serversets are commonly used by Finagle and Aurora.

Usage

discovery.serverset "LABEL" {
	servers = SERVERS_LIST
	paths   = ZOOKEEPER_PATHS_LIST
}

Serverset data stored in Zookeeper must be in JSON format. The Thrift format is not supported.

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`servers`	`list(string)`	The Zookeeper servers to connect to.		yes
`paths`	`list(string)`	The Zookeeper paths to discover Serversets from.		yes
`timeout`	`duration`	The Zookeeper session timeout	`10s`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered.

The following metadata labels are available on targets during relabeling:

__meta_serverset_path: the full path to the serverset member node in Zookeeper
__meta_serverset_endpoint_host: the host of the default endpoint
__meta_serverset_endpoint_port: the port of the default endpoint
__meta_serverset_endpoint_host_<endpoint>: the host of the given endpoint
__meta_serverset_endpoint_port_<endpoint>: the port of the given endpoint
__meta_serverset_shard: the shard number of the member
__meta_serverset_status: the status of the member

Component health

discovery.serverset is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.serverset does not expose any component-specific debug information.

Debug metrics

discovery.serverset does not expose any component-specific debug metrics.

Example

The configuration below will connect to one of the Zookeeper servers (either zk1, zk2, or zk3) and discover JSON Serversets at paths /path/to/znode1 and /path/to/znode2. The discovered targets are scraped by the prometheus.scrape.default component and forwarded to the prometheus.remote_write.default component, which will send the samples to specified remote_write URL.

discovery.serverset "zookeeper" {
	servers = ["zk1", "zk2", "zk3"]
	paths   = ["/path/to/znode1", "/path/to/znode2"]
	timeout = "30s"
}

prometheus.scrape "default" {
	targets    = discovery.serverset.zookeeper.targets
	forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
	endpoint {
		url = "http://remote-write-url1"
	}
}

Compatible components

discovery.serverset has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.triton

Fri, 10 Apr 2026 07:58:08 +0000

discovery.triton

discovery.triton discovers Triton Container Monitors and exposes them as targets.

Usage

discovery.triton "LABEL" {
	account    = ACCOUNT
	dns_suffix = DNS_SUFFIX
	endpoint   = ENDPOINT
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`account`	`string`	The account to use for discovering new targets.		yes
`role`	`string`	The type of targets to discover.	`"container"`	no
`dns_suffix`	`string`	The DNS suffix that is applied to the target.		yes
`endpoint`	`string`	The Triton discovery endpoint.		yes
`groups`	`list(string)`	A list of groups to retrieve targets from.		no
`port`	`int`	The port to use for discovery and metrics scraping.	`9163`	no
`refresh_interval`	`duration`	The refresh interval for the list of targets.	`60s`	no
`version`	`int`	The Triton discovery API version.	`1`	no

role can be set to:

"container" to discover virtual machines (SmartOS zones, lx/KVM/bhyve branded zones) running on Triton
"cn" to discover compute nodes (servers/global zones) making up the Triton infrastructure

groups is only supported when role is set to "container". If omitted all containers owned by the requesting account are scraped.

Blocks

The following blocks are supported inside the definition of discovery.triton:

Hierarchy	Block	Description	Required
tls_config	tls_config	TLS configuration for requests to the Triton API.	no

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Triton API.

When role is set to "container", each target includes the following labels:

__meta_triton_groups: The list of groups belonging to the target joined by a comma separator.
__meta_triton_machine_alias: The alias of the target container.
__meta_triton_machine_brand: The brand of the target container.
__meta_triton_machine_id: The UUID of the target container.
__meta_triton_machine_image: The target container’s image type.
__meta_triton_server_id: The server UUID the target container is running on.

When role is set to "cn" each target includes the following labels:

__meta_triton_machine_alias: The hostname of the target (requires triton-cmon 1.7.0 or newer).
__meta_triton_machine_id: The UUID of the target.

Component health

discovery.triton is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.triton does not expose any component-specific debug information.

Debug metrics

discovery.triton does not expose any component-specific debug metrics.

Example

discovery.triton "example" {
	account    = TRITON_ACCOUNT
	dns_suffix = TRITON_DNS_SUFFIX
	endpoint   = TRITON_ENDPOINT
}

prometheus.scrape "demo" {
	targets    = discovery.triton.example.targets
	forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
	endpoint {
		url = PROMETHEUS_REMOTE_WRITE_URL

		basic_auth {
			username = USERNAME
			password = PASSWORD
		}
	}
}

Replace the following:

TRITON_ACCOUNT: Your Triton account.
TRITON_DNS_SUFFIX: Your Triton DNS suffix.
TRITON_ENDPOINT: Your Triton endpoint.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.triton has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

discovery.uyuni

Fri, 10 Apr 2026 07:58:08 +0000

discovery.uyuni

discovery.uyuni discovers Uyuni Monitoring Endpoints and exposes them as targets.

Usage

discovery.uyuni "LABEL" {
    server   = SERVER
    username = USERNAME
    password = PASSWORD
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`server`	`string`	The primary Uyuni Server.		yes
`username`	`string`	The username to use for authentication to the Uyuni API.		yes
`password`	`Secret`	The password to use for authentication to the Uyuni API.		yes
`entitlement`	`string`	The entitlement to filter on when listing targets.	`"monitoring_entitled"`	no
`separator`	`string`	The separator to use when building the `__meta_uyuni_groups` label.	`","`	no
`refresh_interval`	`duration`	Interval at which to refresh the list of targets.	`1m`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

Blocks

The following blocks are supported inside the definition of discovery.uyuni:

Hierarchy	Block	Description	Required
tls_config	tls_config	TLS configuration for requests to the Uyuni API.	no

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the Uyuni API.

Each target includes the following labels:

__meta_uyuni_minion_hostname: The hostname of the Uyuni Minion.
__meta_uyuni_primary_fqdn: The FQDN of the Uyuni primary.
__meta_uyuni_system_id: The system ID of the Uyuni Minion.
__meta_uyuni_groups: The groups the Uyuni Minion belongs to.
__meta_uyuni_endpoint_name: The name of the endpoint.
__meta_uyuni_exporter: The name of the exporter.
__meta_uyuni_proxy_module: The name of the Uyuni module.
__meta_uyuni_metrics_path: The path to the metrics endpoint.
__meta_uyuni_scheme: https if TLS is enabled on the endpoint, http otherwise.

These labels are largely derived from a listEndpoints API call to the Uyuni Server.

Component health

discovery.uyuni is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

discovery.uyuni does not expose any component-specific debug information.

Debug metrics

discovery.uyuni does not expose any component-specific debug metrics.

Example

discovery.uyuni "example" {
  server    = "https://127.0.0.1/rpc/api"
  username  = UYUNI_USERNAME
  password  = UYUNI_PASSWORD
}

prometheus.scrape "demo" {
  targets    = discovery.uyuni.example.targets
  forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
      username = USERNAME
      password = PASSWORD
    }
  }
}

Replace the following:

UYUNI_USERNAME: The username to use for authentication to the Uyuni server.
UYUNI_PASSWORD: The password to use for authentication to the Uyuni server.
PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Compatible components

discovery.uyuni has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

faro.receiver

Fri, 10 Apr 2026 07:58:08 +0000

faro.receiver

faro.receiver accepts web application telemetry data from the Grafana Faro Web SDK and forwards it to other components for future processing.

Usage

faro.receiver "LABEL" {
    output {
        logs   = [LOKI_RECEIVERS]
        traces = [OTELCOL_COMPONENTS]
    }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`extra_log_labels`	`map(string)`	Extra labels to attach to emitted log lines.	`{}`	no

Blocks

The following blocks are supported inside the definition of faro.receiver:

Hierarchy	Block	Description	Required
server	server	Configures the HTTP server.	no
server > rate_limiting	rate_limiting	Configures rate limiting for the HTTP server.	no
sourcemaps	sourcemaps	Configures sourcemap retrieval.	no
sourcemaps > location	location	Configures on-disk location for sourcemap retrieval.	no
output	output	Configures where to send collected telemetry data.	yes

server block

The server block configures the HTTP server managed by the faro.receiver component. Clients using the Grafana Faro Web SDK forward telemetry data to this HTTP server for processing.

Name	Type	Description	Default	Required
`listen_address`	`string`	Address to listen for HTTP traffic on.	`127.0.0.1`	no
`listen_port`	`number`	Port to listen for HTTP traffic on.	`12347`	no
`cors_allowed_origins`	`list(string)`	Origins for which cross-origin requests are permitted.	`[]`	no
`api_key`	`secret`	Optional API key to validate client requests with.	`""`	no
`max_allowed_payload_size`	`string`	Maximum size (in bytes) for client requests.	`"5MiB"`	no
`include_metadata`	`boolean`	Propagate incoming connection metadata to downstream consumers.	`false`	no

By default, telemetry data is only accepted from applications on the same local network as the browser. To accept telemetry data from a wider set of clients, modify the listen_address attribute to the IP address of the appropriate network interface to use.

The cors_allowed_origins argument determines what origins browser requests may come from. The default value, [], disables CORS support. To support requests from all origins, set cors_allowed_origins to ["*"]. The * character indicates a wildcard.

When the api_key argument is non-empty, client requests must have an HTTP header called X-API-Key matching the value of the api_key argument. Requests that are missing the header or have the wrong value are rejected with an HTTP 401 Unauthorized status code. If the api_key argument is empty, no authentication checks are performed, and the X-API-Key HTTP header is ignored.

rate_limiting block

The rate_limiting block configures rate limiting for client requests.

Name	Type	Description	Default	Required
`enabled`	`bool`	Whether to enable rate limiting.	`true`	no
`rate`	`number`	Rate of allowed requests per second.	`50`	no
`burst_size`	`number`	Allowed burst size of requests.	`100`	no

Rate limiting functions as a token bucket algorithm, where a bucket has a maximum capacity for up to burst_size requests and refills at a rate of rate per second.

Each HTTP request drains the capacity of the bucket by one. Once the bucket is empty, HTTP requests are rejected with an HTTP 429 Too Many Requests status code until the bucket has more available capacity.

Configuring the rate argument determines how fast the bucket refills, and configuring the burst_size argument determines how many requests can be received in a burst before the bucket is empty and starts rejecting requests.

sourcemaps block

The sourcemaps block configures how to retrieve sourcemaps. Sourcemaps are then used to transform file and line information from minified code into the file and line information from the original source code.

Name	Type	Description	Default	Required
`download`	`bool`	Whether to download sourcemaps.	`true`	no
`download_from_origins`	`list(string)`	Which origins to download sourcemaps from.	`["*"]`	no
`download_timeout`	`duration`	Timeout when downloading sourcemaps.	`"1s"`	no

When exceptions are sent to the faro.receiver component, it can download sourcemaps from the web application. You can disable this behavior by setting the download argument to false.

The download_from_origins argument determines which origins a sourcemap may be downloaded from. The origin is attached to the URL that a browser is sending telemetry data from. The default value, ["*"], enables downloading sourcemaps from all origins. The * character indicates a wildcard.

By default, sourcemap downloads are subject to a timeout of "1s", specified by the download_timeout argument. Setting download_timeout to "0s" disables timeouts.

To retrieve sourcemaps from disk instead of the network, specify one or more location blocks. When location blocks are provided, they are checked first for sourcemaps before falling back to downloading.

location block

The location block declares a location where sourcemaps are stored on the filesystem. The location block can be specified multiple times to declare multiple locations where sourcemaps are stored.

Name	Type	Description	Default	Required
`path`	`string`	The path on disk where sourcemaps are stored.		yes
`minified_path_prefix`	`string`	The prefix of the minified path sent from browsers.		yes

The minified_path_prefix argument determines the prefix of paths to Javascript files, such as http://example.com/. The path argument then determines where to find the sourcemap for the file.

For example, given the following location block:

location {
    path                 = "/var/my-app/build"
    minified_path_prefix = "http://example.com/"
}

To look up the sourcemaps for a file hosted at http://example.com/foo.js, the faro.receiver component will:

Remove the minified path prefix to extract the path to the file (foo.js).
Search for that file path with a .map extension (foo.js.map) in path (/var/my-app/build/foo.js.map).

Optionally, the value for the path argument may contain {{ .Release }} as a template value, such as /var/my-app/{{ .Release }}/build. The template value will be replaced with the release value provided by the Faro Web App SDK.

output block

The output block specifies where to forward collected logs and traces.

Name	Type	Description	Default	Required
`logs`	`list(LogsReceiver)`	A list of `loki` components to forward logs to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	A list of `otelcol` components to forward traces to.	`[]`	no

Exported fields

faro.receiver does not export any fields.

Component health

faro.receiver is reported as unhealthy when the integrated server fails to start.

Debug information

faro.receiver does not expose any component-specific debug information.

Debug metrics

faro.receiver exposes the following metrics for monitoring the component:

faro_receiver_logs_total (counter): Total number of ingested logs.
faro_receiver_measurements_total (counter): Total number of ingested measurements.
faro_receiver_exceptions_total (counter): Total number of ingested exceptions.
faro_receiver_events_total (counter): Total number of ingested events.
faro_receiver_exporter_errors_total (counter): Total number of errors produced by an internal exporter.
faro_receiver_request_duration_seconds (histogram): Time (in seconds) spent serving HTTP requests.
faro_receiver_request_message_bytes (histogram): Size (in bytes) of HTTP requests received from clients.
faro_receiver_response_message_bytes (histogram): Size (in bytes) of HTTP responses sent to clients.
faro_receiver_inflight_requests (gauge): Current number of inflight requests.
faro_receiver_sourcemap_cache_size (counter): Number of items in sourcemap cache per origin.
faro_receiver_sourcemap_downloads_total (counter): Total number of sourcemap downloads performed per origin and status.
faro_receiver_sourcemap_file_reads_total (counter): Total number of sourcemap retrievals using the filesystem per origin and status.

Example

faro.receiver "default" {
    server {
        listen_address = "NETWORK_ADDRESS"
    }

    sourcemaps {
        location {
            path                 = "PATH_TO_SOURCEMAPS"
            minified_path_prefix = "WEB_APP_PREFIX"
        }
    }

    output {
        logs   = [loki.write.default.receiver]
        traces = [otelcol.exporter.otlp.traces.input]
    }
}

loki.write "default" {
    endpoint {
        url = "https://LOKI_ADDRESS/api/v1/push"
    }
}

otelcol.exporter.otlp "traces" {
    client {
        endpoint = "OTLP_ADDRESS"
    }
}

Replace the following:

NETWORK_ADDRESS: IP address of the network interface to listen to traffic on. This IP address must be reachable by browsers using the web application to instrument.
PATH_TO_SOURCEMAPS: Path on disk where sourcemaps are located.
WEB_APP_PREFIX: Prefix of the web application being instrumented.
LOKI_ADDRESS: Address of the Loki server to send logs to.
- If authentication is required to send logs to the Loki server, refer to the documentation of loki.write for more information.
OTLP_ADDRESS: The address of the OTLP-compatible server to send traces to.
- If authentication is required to send logs to the Loki server, refer to the documentation of otelcol.exporter.otlp for more information.

Compatible components

faro.receiver can accept arguments from the following components:

Components that export Loki LogsReceiver
Components that export OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

local.file

Fri, 10 Apr 2026 07:58:08 +0000

local.file

local.file exposes the contents of a file on disk to other components. The file will be watched for changes so that its latest content is always exposed.

The most common use of local.file is to load secrets (e.g., API keys) from files.

Multiple local.file components can be specified by giving them different labels.

Usage

local.file "LABEL" {
  filename = FILE_NAME
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`filename`	`string`	Path of the file on disk to watch		yes
`detector`	`string`	Which file change detector to use (fsnotify, poll)	`"fsnotify"`	no
`poll_frequency`	`duration`	How often to poll for file changes	`"1m"`	no
`is_secret`	`bool`	Marks the file as containing a secret	`false`	no

File change detectors

File change detectors detect when the file needs to be re-read from disk. local.file supports two detectors: fsnotify and poll.

fsnotify

The fsnotify detector subscribes to filesystem events, which indicate when the watched file is updated. This detector requires a filesystem that supports events at the operating system level. Network-based filesystems like NFS or FUSE won’t work.

The component re-reads the watched file when a filesystem event is received. This re-read happens for any filesystem event related to the file, including a permissions change.

fsnotify also polls for changes to the file with the configured poll_frequency as a fallback.

fsnotify stops receiving filesystem events if the watched file has been deleted, renamed, or moved. The subscription is re-established on the next poll once the watched file exists again.

poll

The poll file change detector causes the watched file to be re-read every poll_frequency, regardless of whether the file changed.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`content`	`string` or `secret`	The contents of the file from the most recent read

The content field will have the secret type only if the is_secret argument was true.

Component health

local.file will be reported as healthy whenever if the watched file was read successfully.

Failing to read the file whenever an update is detected (or after the poll period elapses) will cause the component to be reported as unhealthy. When unhealthy, exported fields will be kept at the last healthy value. The read error will be exposed as a log message and in the debug information for the component.

Debug information

local.file does not expose any component-specific debug information.

Debug metrics

agent_local_file_timestamp_last_accessed_unix_seconds (gauge): The timestamp, in Unix seconds, that the file was last successfully accessed.

Example

local.file "secret_key" {
  filename  = "/var/secrets/password.txt"
  is_secret = true
}

local.file_match

Fri, 10 Apr 2026 07:58:08 +0000

local.file_match

local.file_match discovers files on the local filesystem using glob patterns and the doublestar library.

Usage

local.file_match "LABEL" {
  path_targets = [{"__path__" = DOUBLESTAR_PATH}]
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`path_targets`	`list(map(string))`	Targets to expand; looks for glob patterns on the `__path__` and `__path_exclude__` keys.		yes
`sync_period`	`duration`	How often to sync filesystem and targets.	`"10s"`	no

path_targets uses doublestar style paths.

/tmp/**/*.log will match all subfolders of tmp and include any files that end in *.log.
/tmp/apache/*.log will match only files in /tmp/apache/ that end in *.log.
/tmp/** will match all subfolders of tmp, tmp itself, and all files.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`targets`	`list(map(string))`	The set of targets discovered from the filesystem.

Each target includes the following labels:

__path__: Absolute path to the file.

Component health

local.file_match is only reported as unhealthy when given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

local.file_match does not expose any component-specific debug information.

Debug metrics

local.file_match does not expose any component-specific debug metrics.

Examples

Send `/tmp/logs/*.log` files to Loki

This example discovers all files and folders under /tmp/logs. The absolute paths are used by loki.source.file.files targets.

local.file_match "tmp" {
  path_targets = [{"__path__" = "/tmp/logs/**/*.log"}]
}

loki.source.file "files" {
  targets    = local.file_match.tmp.targets
  forward_to = [loki.write.endpoint.receiver]
}

loki.write "endpoint" {
  endpoint {
      url = LOKI_URL
      basic_auth {
          username = USERNAME
          password = PASSWORD
      }
  }
}

Replace the following:

LOKI_URL: The URL of the Loki server to send logs to.
USERNAME: The username to use for authentication to the Loki API.
PASSWORD: The password to use for authentication to the Loki API.

Send Kubernetes pod logs to Loki

This example finds all the logs on pods and monitors them.

discovery.kubernetes "k8s" {
  role = "pod"
}

discovery.relabel "k8s" {
  targets = discovery.kubernetes.k8s.targets

  rule {
    source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_label_name"]
    target_label  = "job"
    separator     = "/"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_uid", "__meta_kubernetes_pod_container_name"]
    target_label  = "__path__"
    separator     = "/"
    replacement   = "/var/log/pods/*$1/*.log"
  }
}

local.file_match "pods" {
  path_targets = discovery.relabel.k8s.output
}

loki.source.file "pods" {
  targets = local.file_match.pods.targets
  forward_to = [loki.write.endpoint.receiver]
}

loki.write "endpoint" {
  endpoint {
      url = LOKI_URL
      basic_auth {
          username = USERNAME
          password = PASSWORD
      }
  }
}

Replace the following:

LOKI_URL: The URL of the Loki server to send logs to.
USERNAME: The username to use for authentication to the Loki API.
PASSWORD: The password to use for authentication to the Loki API.

Compatible components

local.file_match can accept arguments from the following components:

Components that export Targets

local.file_match has exports that can be consumed by the following components:

Components that consume Targets

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.echo

Fri, 10 Apr 2026 07:58:08 +0000

loki.echo

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

loki.echo receives log entries from other loki components and prints them to the process’ standard output (stdout).

Multiple loki.echo components can be specified by giving them different labels.

Usage

loki.echo "LABEL" {}

Arguments

loki.echo accepts no arguments.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`LogsReceiver`	A value that other components can use to send log entries to.

Component health

loki.echo is only reported as unhealthy if given an invalid configuration.

Debug information

loki.echo does not expose any component-specific debug information.

Example

This example creates a pipeline that reads log files from /var/log and prints log lines to echo:

local.file_match "varlog" {
  path_targets = [{
    __path__ = "/var/log/*log",
    job      = "varlog",
  }]
}

loki.source.file "logs" {
  targets    = local.file_match.varlog.targets
  forward_to = [loki.echo.example.receiver]
}

loki.echo "example" { }

Compatible components

loki.echo has exports that can be consumed by the following components:

Components that consume Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.process

Fri, 10 Apr 2026 07:58:08 +0000

loki.process

loki.process receives log entries from other loki components, applies one or more processing stages, and forwards the results to the list of receivers in the component’s arguments.

A stage is a multi-purpose tool that can parse, transform, and filter log entries before they’re passed to a downstream component. These stages are applied to each log entry in order of their appearance in the configuration file. All stages within a loki.process block have access to the log entry’s label set, the log line, the log timestamp, as well as a shared map of ’extracted’ values so that the results of one stage can be used in a subsequent one.

Multiple loki.process components can be specified by giving them different labels.

Usage

loki.process "LABEL" {
  forward_to = RECEIVER_LIST

  stage.STAGENAME {
    ...
  }
  ...
}

Arguments

loki.process supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	Where to forward log entries after processing.		yes

Blocks

The following blocks are supported inside the definition of loki.process:

Hierarchy	Block	Description	Required
stage.cri	stage.cri	Configures a pre-defined CRI-format pipeline.	no
stage.decolorize	stage.decolorize	Strips ANSI color codes from log lines.	no
stage.docker	stage.docker	Configures a pre-defined Docker log format pipeline.	no
stage.drop	stage.drop	Configures a `drop` processing stage.	no
stage.eventlogmessage	stage.eventlogmessage	Extracts data from the Message field in the Windows Event Log.	no
stage.geoip	stage.geoip	Configures a `geoip` processing stage.	no
stage.json	stage.json	Configures a JSON processing stage.	no
stage.label_drop	stage.label_drop	Configures a `label_drop` processing stage.	no
stage.label_keep	stage.label_keep	Configures a `label_keep` processing stage.	no
stage.labels	stage.labels	Configures a `labels` processing stage.	no
stage.limit	stage.limit	Configures a `limit` processing stage.	no
stage.logfmt	stage.logfmt	Configures a `logfmt` processing stage.	no
stage.luhn	stage.luhn	Configures a `luhn` processing stage.	no
stage.match	stage.match	Configures a `match` processing stage.	no
stage.metrics	stage.metrics	Configures a `metrics` stage.	no
stage.multiline	stage.multiline	Configures a `multiline` processing stage.	no
stage.output	stage.output	Configures an `output` processing stage.	no
stage.pack	stage.pack	Configures a `pack` processing stage.	no
stage.regex	stage.regex	Configures a `regex` processing stage.	no
stage.replace	stage.replace	Configures a `replace` processing stage.	no
stage.sampling	stage.sampling	Samples logs at a given rate.	no
stage.static_labels	stage.static_labels	Configures a `static_labels` processing stage.	no
stage.structured_metadata	stage.structured_metadata	Configures a structured metadata processing stage.	no
stage.template	stage.template	Configures a `template` processing stage.	no
stage.tenant	stage.tenant	Configures a `tenant` processing stage.	no
stage.timestamp	stage.timestamp	Configures a `timestamp` processing stage.	no

A user can provide any number of these stage blocks nested inside loki.process; these will run in order of appearance in the configuration file.

stage.cri block

The stage.cri inner block enables a predefined pipeline which reads log lines using the CRI logging format.

The following arguments are supported:

Name	Type	Description	Default	Required
`max_partial_lines`	`number`	Maximum number of partial lines to hold in memory.	`100`	no
`max_partial_line_size`	`number`	Maximum number of characters which a partial line can have.	`0`	no
`max_partial_line_size_truncate`	`bool`	Truncate partial lines that are longer than `max_partial_line_size`.	`false`	no

max_partial_line_size is only taken into account if max_partial_line_size_truncate is set to true.

stage.cri {}

CRI specifies log lines as single space-delimited values with the following components:

time: The timestamp string of the log
stream: Either stdout or stderr
flags: CRI flags including F or P
log: The contents of the log line

Given the following log line, the subsequent key-value pairs are created in the shared map of extracted data:

"2019-04-30T02:12:41.8443515Z stdout F message"

content: message
stream: stdout
timestamp: 2019-04-30T02:12:41.8443515

stage.decolorize block

The stage.decolorize strips ANSI color codes from the log lines, thus making it easier to parse logs further.

The stage.decolorize block does not support any arguments or inner blocks, so it is always empty.

stage.decolorize {}

stage.decolorize turns each line having a color code into a non-colored one, for example:

[2022-11-04 22:17:57.811] \033[0;32http\033[0m: GET /_health (0 ms) 204

is turned into

[2022-11-04 22:17:57.811] http: GET /_health (0 ms) 204

stage.docker block

The stage.docker inner block enables a predefined pipeline which reads log lines in the standard format of Docker log files.

The stage.docker block does not support any arguments or inner blocks, so it is always empty.

stage.docker {}

Docker log entries are formatted as JSON with the following keys:

log: The content of log line
stream: Either stdout or stderr
time: The timestamp string of the log line

Given the following log line, the subsequent key-value pairs are created in the shared map of extracted data:

{"log":"log message\n","stream":"stderr","time":"2019-04-30T02:12:41.8443515Z"}

output: log message\n
stream: stderr
timestamp: 2019-04-30T02:12:41.8443515

stage.drop block

The stage.drop inner block configures a filtering stage that drops log entries based on several options. If multiple options are provided, they’re treated as AND clauses and must all be true for the log entry to be dropped. To drop entries with an OR clause, specify multiple drop blocks in sequence.

The following arguments are supported:

Name	Type	Description	Default	Required
`source`	`string`	Name or comma-separated list of names from extracted data to match. If empty or not defined, it uses the log message.	`""`	no
`separator`	`string`	When `source` is a comma-separated list of names, this separator is placed between concatenated extracted data values.	`";"`	no
`expression`	`string`	A valid RE2 regular expression.	`""`	no
`value`	`string`	If both `source` and `value` are specified, the stage drops lines where `value` exactly matches the source content.	`""`	no
`older_than`	`duration`	If specified, the stage drops lines whose timestamp is older than the current time minus this duration.	`""`	no
`longer_than`	`string`	If specified, the stage drops lines whose size exceeds the configured value.	`""`	no
`drop_counter_reason`	`string`	A custom reason to report for dropped lines.	`"drop_stage"`	no

The expression field must be a RE2 regex string.

If source is empty or not provided, the regex attempts to match the log line itself.
If source is a single name, the regex attempts to match the corresponding value from the extracted map.
If source is a comma-separated list of names, the corresponding values from the extracted map are concatenated using separator and the regex attempts to match the concatenated string.

The value field can only work with values from the extracted map, and must be specified together with source.

If source is a single name, the entries are dropped when there is an exact match between the corresponding value from the extracted map and the value.
If source is a comma-separated list of names, the entries are dropped when the value matches the source values from extracted data, concatenated using the separator.

Whenever an entry is dropped, the metric loki_process_dropped_lines_total is incremented. By default, the reason label is "drop_stage", but you can provide a custom label using the drop_counter_reason argument.

The following stage drops log entries that contain the word debug and are longer than 1KB.

stage.drop {
    expression  = ".*debug.*"
    longer_than = "1KB"
}

On the following example, we define multiple drop blocks so loki.process drops entries that are either 24h or older, are longer than 8KB, or the extracted value of ‘app’ is equal to foo.

stage.drop {
    older_than          = "24h"
    drop_counter_reason = "too old"
}

stage.drop {
    longer_than         = "8KB"
    drop_counter_reason = "too long"
}

stage.drop {
    source = "app"
    value  = "foo"
}

stage.eventlogmessage block

The eventlogmessage stage extracts data from the Message string that appears in the Windows Event Log.

The following arguments are supported:

Name	Type	Description	Default	Required
`source`	`string`	Name of the field in the extracted data to parse.	`message`	no
`overwrite_existing`	`bool`	Whether to overwrite existing extracted data fields.	`false`	no
`drop_invalid_labels`	`bool`	Whether to drop fields that are not valid label names.	`false`	no

When overwrite_existing is set to true, the stage overwrites existing extracted data fields with the same name. If set to false, the _extracted suffix will be appended to an already existing field name.

When drop_invalid_labels is set to true, the stage drops fields that are not valid label names. If set to false, the stage will automatically convert them into valid labels replacing invalid characters with underscores.

Example combined with `stage.json`

stage.json {
    expressions = { 
        message = "", 
        Overwritten = "",
    }
}

stage.eventlogmessage {
    source = "message"
    overwrite_existing = true
}

Given the following log line:

{"event_id": 1, "Overwritten": "old", "message": "Message type:\r\nOverwritten: new\r\nImage: C:\\Users\\User\\agent.exe"}

The first stage would create the following key-value pairs in the set of extracted data:

message: Message type:\r\nOverwritten: new\r\nImage: C:\Users\User\agent.exe
Overwritten: old

The second stage will parse the value of message from the extracted data and append/overwrite the following key-value pairs to the set of extracted data:

Image: C:\\Users\\User\\agent.exe
Message_type: (empty string)
Overwritten: new

stage.json block

The stage.json inner block configures a JSON processing stage that parses incoming log lines or previously extracted values as JSON and uses JMESPath expressions to extract new values from them.

The following arguments are supported:

Name	Type	Description	Default	Required
`expressions`	`map(string)`	Key-value pairs of JMESPath expressions.		yes
`source`	`string`	Source of the data to parse as JSON.	`""`	no
`drop_malformed`	`bool`	Drop lines whose input cannot be parsed as valid JSON.	`false`	no

When configuring a JSON stage, the source field defines the source of data to parse as JSON. By default, this is the log line itself, but it can also be a previously extracted value.

The expressions field is the set of key-value pairs of JMESPath expressions to run. The map key defines the name with which the data is extracted, while the map value is the expression used to populate the value.

Here’s a given log line and two JSON stages to run.

{"log":"log message\n","extra":"{\"user\":\"agent\"}"}

loki.process "username" {
  stage.json {
      expressions = {output = "log", extra = ""}
  }

  stage.json {
      source      = "extra"
      expressions = {username = "user"}
  }
}

In this example, the first stage uses the log line as the source and populates these values in the shared map. An empty expression means using the same value as the key (so extra="extra").

output: log message\n
extra: {"user": "agent"}

The second stage uses the value in extra as the input and appends the following key-value pair to the set of extracted data.

username: agent

Note
Due to a limitation of the upstream jmespath library, you must wrap any string that contains a hyphen - in quotes so that it’s not considered a numerical expression.

If you don’t use quotes to wrap a string that contains a hyphen, you will get errors like: Unexpected token at the end of the expression: tNumber

You can use one of two options to circumvent this issue:

An escaped double quote. For example: http_user_agent = "\"request_User-Agent\""

A backtick quote. For example: http_user_agent = `"request_User-Agent"`

stage.label_drop block

The stage.label_drop inner block configures a processing stage that drops labels from incoming log entries.

The following arguments are supported:

Name	Type	Description	Default	Required
`values`	`list(string)`	Configures a `label_drop` processing stage.	`{}`	no

stage.label_drop {
    values = [ "kubernetes_node_name", "kubernetes_namespace" ]
}

stage.label_keep block

The stage.label_keep inner block configures a processing stage that filters the label set of an incoming log entry down to a subset.

The following arguments are supported:

Name	Type	Description	Default	Required
`values`	`list(string)`	Configures a `label_keep` processing stage.	`{}`	no

stage.label_keep {
    values = [ "kubernetes_pod_name", "kubernetes_pod_container_name" ]
}

stage.labels block

The stage.labels inner block configures a labels processing stage that can read data from the extracted values map and set new labels on incoming log entries.

The following arguments are supported:

Name	Type	Description	Default	Required
`values`	`map(string)`	Configures a `labels` processing stage.	`{}`	no

In a labels stage, the map’s keys define the label to set and the values are how to look them up. If the value is empty, it is inferred to be the same as the key.

stage.labels {
    values = {
      env  = "",         // Sets up an 'env' label, based on the 'env' extracted value.
      user = "username", // Sets up a 'user' label, based on the 'username' extracted value.
    }
}

stage.structured_metadata block

The stage.structured_metadata inner block configures a stage that can read data from the extracted values map and add them to log entries as structured metadata.

The following arguments are supported:

Name	Type	Description	Default	Required
`values`	`map(string)`	Specifies the list of labels to add from extracted values map to log entry.	`{}`	no

In a structured_metadata stage, the map’s keys define the label to set and the values are how to look them up. If the value is empty, it is inferred to be the same as the key.

stage.structured_metadata {
    values = {
      env  = "",         // Sets up an 'env' property to structured metadata, based on the 'env' extracted value.
      user = "username", // Sets up a 'user' property to structured metadata, based on the 'username' extracted value.
    }
}

stage.limit block

The stage.limit inner block configures a rate-limiting stage that throttles logs based on several options.

The following arguments are supported:

Name	Type	Description	Default	Required
`rate`	`number`	The maximum rate of lines per second that the stage forwards.		yes
`burst`	`number`	The maximum number of burst lines that the stage forwards.		yes
`by_label_name`	`string`	The label to use when rate-limiting on a label name.	`""`	no
`drop`	`bool`	Whether to discard or backpressure lines that exceed the rate limit.	`false`	no
`max_distinct_labels`	`number`	The number of unique values to keep track of when rate-limiting `by_label_name`.	`10000`	no

The rate limiting is implemented as a “token bucket” of size burst, initially full and refilled at rate tokens per second. Each received log entry consumes one token from the bucket. When drop is set to true, incoming entries that exceed the rate-limit are dropped, otherwise they are queued until more tokens are available.

stage.limit {
    rate  = 5
    burst = 10
}

If by_label_name is set, then drop must be set to true. This enables the stage to rate-limit not by the number of lines but by the number of labels.

The following example rate-limits entries from each unique namespace value independently. Any entries without the namespace label are not rate-limited. The stage keeps track of up to max_distinct_labels unique values, defaulting at 10000.

stage.limit {
    rate  = 10
    burst = 10
    drop  = true

    by_label_name = "namespace"
}

stage.logfmt block

The stage.logfmt inner block configures a processing stage that reads incoming log lines as logfmt and extracts values from them.

The following arguments are supported:

Name	Type	Description	Default	Required
`mapping`	`map(string)`	Key-value pairs of logmft fields to extract.		yes
`source`	`string`	Source of the data to parse as logfmt.	`""`	no

The source field defines the source of data to parse as logfmt. When source is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.

This stage uses the go-logfmt unmarshaler, so that numeric or boolean types are unmarshalled into their correct form. The stage does not perform any other type conversions. If the extracted value is a complex type, it is treated as a string.

Let’s see how this works on the following log line and stages.

time=2012-11-01T22:08:41+00:00 app=loki level=WARN duration=125 message="this is a log line" extra="user=foo"

stage.logfmt {
    mapping = { "extra" = "" }
}

stage.logfmt {
    mapping = { "username" = "user" }
    source  = "extra"
}

The first stage parses the log line itself and inserts the extra key in the set of extracted data, with the value of user=foo.

The second stage parses the contents of extra and appends the username: foo key-value pair to the set of extracted data.

stage.luhn block

The stage.luhn inner block configures a processing stage that reads incoming log lines and redacts strings that match a Luhn algorithm.

The Luhn algorithm is a simple checksum formula used to validate various identification numbers, such as credit card numbers, IMEI numbers, National Provider Identifier numbers in the US, and Canadian Social Insurance Numbers. Many Payment Card Industry environments require these numbers to be redacted.

The following arguments are supported:

Name	Type	Description	Default	Required
`replacement`	`string`	String to substitute the matched patterns with	`"REDACTED"`	no
`source`	`string`	Source of the data to parse.	`""`	no
`minLength`	`int`	Minimum length of digits to consider	`13`	no

The source field defines the source of data to search. When source is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value.

The following example log line contains an approved credit card number.

time=2012-11-01T22:08:41+00:00 app=loki level=WARN duration=125 message="credit card approved 4032032513548443" extra="user=foo"

stage.luhn {
    replacement = "**DELETED**"
}

The stage parses the log line, redacts the credit card number, and produces the following updated log line:

time=2012-11-01T22:08:41+00:00 app=loki level=INFO duration=125 message="credit card approved **DELETED**" extra="user=foo"

stage.match block

The stage.match inner block configures a filtering stage that can conditionally either apply a nested set of processing stages or drop an entry when a log entry matches a configurable LogQL stream selector and filter expressions.

The following arguments are supported:

Name	Type	Description	Default	Required
`selector`	`string`	The LogQL stream selector and line filter expressions to use.		yes
`pipeline_name`	`string`	A custom name to use for the nested pipeline.	`""`	no
`action`	`string`	The action to take when the selector matches the log line. Supported values are `"keep"` and `"drop"`	`"keep"`	no
`drop_counter_reason`	`string`	A custom reason to report for dropped lines.	`"match_stage"`	no

Note
The filters do not include label filter expressions such as | label == "foobar".

The stage.match block supports a number of stage.* inner blocks, like the top-level block. These are used to construct the nested set of stages to run if the selector matches the labels and content of the log entries. It supports all the same stage.NAME blocks as the in the top level of the loki.process component.

If the specified action is "drop", the metric loki_process_dropped_lines_total is incremented with every line dropped. By default, the reason label is "match_stage", but a custom reason can be provided by using the drop_counter_reason argument.

Let’s see this in action, with the following log lines and stages

{ "time":"2023-01-18T17:08:41+00:00", "app":"foo", "component": ["parser","type"], "level" : "WARN", "message" : "app1 log line" }
{ "time":"2023-01-18T17:08:42+00:00", "app":"bar", "component": ["parser","type"], "level" : "ERROR", "message" : "foo noisy error" }

stage.json {
    expressions = { "appname" = "app" }
}

stage.labels {
    values = { "applbl" = "appname" }
}

stage.match {
    selector = "{applbl=\"foo\"}"

    stage.json {
        expressions = { "msg" = "message" }
    }
}

stage.match {
    selector = "{applbl=\"qux\"}"
    stage.json {
        expressions = { "msg" = "msg" }
    }
}

stage.match {
    selector = "{applbl=\"bar\"} |~ \".*noisy error.*\""
    action   = "drop"

    drop_counter_reason = "discard_noisy_errors"
}

stage.output {
    source = "msg"
}

The first two stages parse the log lines as JSON, decode the app value into the shared extracted map as appname, and use its value as the applbl label.

The third stage uses the LogQL selector to only execute the nested stages on lines where the applbl="foo". So, for the first line, the nested JSON stage adds msg="app1 log line" into the extracted map.

The fourth stage uses the LogQL selector to only execute on lines where applbl="qux"; that means it won’t match any of the input, and the nested JSON stage does not run.

The fifth stage drops entries from lines where applbl is set to ‘bar’ and the line contents matches the regex .*noisy error.*. It also increments the loki_process_dropped_lines_total metric with a label drop_counter_reason="discard_noisy_errors".

The final output stage changes the contents of the log line to be the value of msg from the extracted map. In this case, the first log entry’s content is changed to app1 log line.

stage.metrics block

The stage.metrics inner block configures stage that allows to define and update metrics based on values from the shared extracted map. The created metrics are available at the Agent’s root /metrics endpoint.

The stage.metrics block does not support any arguments and is only configured via a number of nested inner metric.* blocks, one for each metric that should be generated.

The following blocks are supported inside the definition of stage.metrics:

Hierarchy	Block	Description	Required
metric.counter	metric.counter	Defines a `counter` metric.	no
metric.gauge	metric.gauge	Defines a `gauge` metric.	no
metric.histogram	metric.histogram	Defines a `histogram` metric.	no

Note
The metrics will be reset if you reload the Grafana Agent configuration file.

metric.counter block

Defines a metric whose value only goes up.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The metric name.		yes
`action`	`string`	The action to take. Valid actions are `set`, `inc`, `dec`, `add`, or `sub`.		yes
`description`	`string`	The metric’s description and help text.	`""`	no
`source`	`string`	Key from the extracted data map to use for the metric. Defaults to the metric name.	`""`	no
`prefix`	`string`	The prefix to the metric name.	`"loki_process_custom_"`	no
`max_idle_duration`	`duration`	Maximum amount of time to wait until the metric is marked as ‘stale’ and removed.	`"5m"`	no
`value`	`string`	If set, the metric only changes if `source` exactly matches the `value`.	`""`	no
`match_all`	`bool`	If set to true, all log lines are counted, without attemptng to match the `source` to the extracted map.	`false`	no
`count_entry_bytes`	`bool`	If set to true, counts all log lines bytes.	`false`	no

A counter cannot set both match_all to true and a value. A counter cannot set count_entry_bytes without also setting match_all=true or action=add. The valid action values are inc and add. The inc action increases the metric value by 1 for each log line that passed the filter. The add action converts the extracted value to a positive float and adds it to the metric.

metric.gauge block

Defines a gauge metric whose value can go up or down.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The metric name.		yes
`action`	`string`	The action to take. Valid actions are `inc` and `add`.		yes
`description`	`string`	The metric’s description and help text.	`""`	no
`source`	`string`	Key from the extracted data map to use for the metric. Defaults to the metric name.	`""`	no
`prefix`	`string`	The prefix to the metric name.	`"loki_process_custom_"`	no
`max_idle_duration`	`duration`	Maximum amount of time to wait until the metric is marked as ‘stale’ and removed.	`"5m"`	no
`value`	`string`	If set, the metric only changes if `source` exactly matches the `value`.	`""`	no

The valid action values are inc, dec, set, add, or sub. inc and dec increment and decrement the metric’s value by 1 respectively. If set, add, or sub` is chosen, the extracted value must be convertible to a positive float and is set, added to, or subtracted from the metric’s value.

metric.histogram block

Defines a histogram metric whose values are recorded in predefined buckets.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The metric name.		yes
`buckets`	`list(float)`	The action to take. Valid actions are `set`, `inc`, `dec`, `add`, or `sub`.		yes
`description`	`string`	The metric’s description and help text.	`""`	no
`source`	`string`	Key from the extracted data map to use for the metric. Defaults to the metric name.	`""`	no
`prefix`	`string`	The prefix to the metric name.	`"loki_process_custom_"`	no
`max_idle_duration`	`duration`	Maximum amount of time to wait until the metric is marked as ‘stale’ and removed.	`"5m"`	no
`value`	`string`	If set, the metric only changes if `source` exactly matches the `value`.	`""`	no

metrics behavior

If value is not present, all incoming log entries match.

Label values on created metrics can be dynamic, which can cause exported metrics to explode in cardinality or go stale, for example, when a stream stops receiving new logs. To prevent unbounded growth of the /metrics endpoint, any metrics which have not been updated within max_idle_duration are removed. The max_idle_duration must be greater or equal to "1s", and it defaults to "5m".

The metric values extracted from the log data are internally converted to floats. The supported values are the following:

integer
floating point number
string - Two types of string format are supported:
- Strings that represent floating point numbers, for example, “0.804” is converted to 0.804.
- Duration format strings. Valid time units are “ns”, “us”, “ms”, “s”, “m”, “h”. A value in this format is converted to a floating point number of seconds, for example, “0.5ms” is converted to 0.0005.
boolean:
- true is converted to 1.
- false is converted to 0.

The following pipeline creates a counter which increments every time any log line is received by using the match_all parameter. The pipeline creates a second counter which adds the byte size of these log lines by using the count_entry_bytes parameter.

These two metrics disappear after 24 hours if no new entries are received, to avoid building up metrics which no longer serve any use. These two metrics are a good starting point to track the volume of log streams in both the number of entries and their byte size, to identify sources of high-volume or high-cardinality data.

stage.metrics {
    metric.counter {
        name        = "log_lines_total"
        description = "total number of log lines"
        prefix      = "my_custom_tracking_"

        match_all         = true
        action            = "inc"
        max_idle_duration = "24h"
    }
}
stage.metrics {
    metric.counter {
        name        = "log_bytes_total"
        description = "total bytes of log lines"
        prefix      = "my_custom_tracking_"

        match_all         = true
        count_entry_bytes = true
        action            = "add"
        max_idle_duration = "24h"
    }
}

Here, the first stage uses a regex to extract text in the format order_status=<string> in the log line. The second stage, defines a counter which increments the successful_orders_total and failed_orders_total based on the previously extracted values.

stage.regex {
    expression = "^.* order_status=(?P<order_status>.*?) .*$"
}
stage.metrics {
    metric.counter {
        name        = "successful_orders_total"
        description = "successful orders"
        source      = "order_status"
        value       = "success"
        action      = "inc"
    }
}
stage.metrics {
    metric.counter {
        name        = "failed_orders_total"
        description = "failed orders"
        source      = "order_status"
        value       = "fail"
        action      = "inc"
    }
}

In this example, the first stage extracts text in the format of retries=<value>, from the log line. The second stage creates a gauge whose current metric value is increased by the number extracted from the retries field.

stage.regex {
    expression = "^.* retries=(?P<retries>\\d+) .*$"
}
stage.metrics {
    metric.gauge {
        name        = "retries_total"
        description = "total_retries"
        source      = "retries"
        action      = "add"
    }
}

The following example shows a histogram that reads response_time from the extracted map and places it into a bucket, both increasing the count of the bucket and the sum for that particular bucket:

stage.metrics {
    metric.histogram {
        name        = "http_response_time_seconds"
        description = "recorded response times"
        source      = "response_time"
        buckets     = [0.001,0.0025,0.005,0.010,0.025,0.050]
    }
}

stage.multiline block

The stage.multiline inner block merges multiple lines into a single block before passing it on to the next stage in the pipeline.

The following arguments are supported:

Name	Type	Description	Default	Required
`firstline`	`string`	Name from extracted data to use for the log entry.		yes
`max_wait_time`	`duration`	The maximum time to wait for a multiline block.	`"3s"`	no
`max_lines`	`number`	The maximum number of lines a block can have.	`128`	no

A new block is identified by the RE2 regular expression passed in firstline.

Any line that does not match the expression is considered to be part of the block of the previous match. If no new logs arrive with max_wait_time, the block is sent on. The max_lines field defines the maximum number of lines a block can have. If this is exceeded, a new block is started.

Let’s see how this works in practice with an example stage and a stream of log entries from a Flask web service.

stage.multiline {
    firstline     = "^\\[\\d{4}-\\d{2}-\\d{2} \\d{1,2}:\\d{2}:\\d{2}\\]"
    max_wait_time = "10s"
}

[2023-01-18 17:41:21] "GET /hello HTTP/1.1" 200 -
[2023-01-18 17:41:25] ERROR in app: Exception on /error [GET]
Traceback (most recent call last):
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/pallets/src/deployment_tools/hello.py", line 10, in error
    raise Exception("Sorry, this route always breaks")
Exception: Sorry, this route always breaks
[2023-01-18 17:42:24] "GET /error HTTP/1.1" 500 -
[2023-01-18 17:42:29] "GET /hello HTTP/1.1" 200 -

All ‘blocks’ that form log entries of separate web requests start with a timestamp in square brackets. The stage detects this with the regular expression in firstline to collapse all lines of the traceback into a single block and thus a single Loki log entry.

stage.output block

The stage.output inner block configures a processing stage that reads from the extracted map and changes the content of the log entry that is forwarded to the next component.

The following arguments are supported:

Name	Type	Description	Default	Required
`source`	`string`	Name from extracted data to use for the log entry.		yes

Let’s see how this works for the following log line and three-stage pipeline:

{"user": "John Doe", "message": "hello, world!"}

stage.json {
    expressions = { "user" = "user", "message" = "message" }
}

stage.labels {
    values = { "user" = "user" }
}

stage.output {
    source = "message"
}

The first stage extracts the following key-value pairs into the shared map:

user: John Doe
message: hello, world!

Then, the second stage adds user="John Doe" to the label set of the log entry, and the final output stage changes the log line from the original JSON to hello, world!.

stage.pack block

The stage.pack inner block configures a transforming stage that replaces the log entry with a JSON object that embeds extracted values and labels with it.

The following arguments are supported:

Name	Type	Description	Default	Required
`labels`	`list(string)`	The values from the extracted data and labels to pack with the log entry.		yes
`ingest_timestamp`	`bool`	Whether to replace the log entry timestamp with the time the `pack` stage runs.	`true`	no

This stage lets you embed extracted values and labels together with the log line, by packing them into a JSON object. The original message is stored under the _entry key, and all other keys retain their values. This is useful in cases where you do want to keep a certain label or metadata, but you don’t want it to be indexed as a label due to high cardinality.

The querying capabilities of Loki make it easy to still access this data so it can be filtered and aggregated at query time.

For example, consider the following log entry:

log_line: "something went wrong"
labels:   { "level" = "error", "env" = "dev", "user_id" = "f8fas0r" }

and this processing stage:

stage.pack {
    labels = ["env", "user_id"]
}

The stage transforms the log entry into the following JSON object, where the two embedded labels are removed from the original log entry:

{
  "_entry": "something went wrong",
  "env": "dev",
  "user_id": "f8fas0r"
}

At query time, Loki’s unpack parser can be used to access these embedded labels and replace the log line with the original one stored in the _entry field automatically.

When combining several log streams to use with the pack stage, you can set ingest_timestamp to true to avoid interlaced timestamps and out-of-order ingestion issues.

stage.regex block

The stage.regex inner block configures a processing stage that parses log lines using regular expressions and uses named capture groups for adding data into the shared extracted map of values.

The following arguments are supported:

Name	Type	Description	Default	Required
`expression`	`string`	A valid RE2 regular expression. Each capture group must be named.		yes
`source`	`string`	Name from extracted data to parse. If empty, uses the log message.	`""`	no

The expression field needs to be a RE2 regex string. Every matched capture group is added to the extracted map, so it must be named like: (?P<name>re). The name of the capture group is then used as the key in the extracted map for the matched value.

Because of how River strings work, any backslashes in expression must be escaped with a double backslash; for example "\\w" or "\\S+".

If the source is empty or missing, then the stage parses the log line itself. If it’s set, the stage parses a previously extracted value with the same name.

Given the following log line and regex stage, the extracted values are shown below:

2019-01-01T01:00:00.000000001Z stderr P i'm a log message!

stage.regex {
    expression = "^(?s)(?P<time>\\S+?) (?P<stream>stdout|stderr) (?P<flags>\\S+?) (?P<content>.*)$"
}

time: 2019-01-01T01:00:00.000000001Z,
stream: stderr,
flags: P,
content: i'm a log message

On the other hand, if the source value is set, then the regex is applied to the value stored in the shared map under that name.

Let’s see what happens when the following log line is put through this two-stage pipeline:

{"timestamp":"2022-01-01T01:00:00.000000001Z"}

stage.json {
    expressions = { time = "timestamp" }
}
stage.regex {
    expression = "^(?P<year>\\d+)"
    source     = "time"
}

The first stage adds the following key-value pair into the extracted map:

time: 2022-01-01T01:00:00.000000001Z

Then, the regex stage parses the value for time from the shared values and appends the subsequent key-value pair back into the extracted values map:

year: 2022

stage.replace block

The stage.replace inner block configures a stage that parses a log line using a regular expression and replaces the log line contents. Named capture groups in the regex also support adding data into the shared extracted map.

The following arguments are supported:

Name	Type	Description	Required
`expression`	`string`	Name from extracted data to use for the log entry.	yes
`source`	`string`	Source of the data to parse. If empty, it uses the log message.	no
`replace`	`string`	Value replaced by the capture group.	no

The source field defines the source of data to parse using expression. When source is missing or empty, the stage parses the log line itself, but it can also be used to parse a previously extracted value. The replaced value is assigned back to the source key.

The expression must be a valid RE2 regex. Every named capture group (?P<name>re) is set into the extracted map with its name.

Because of how River treats backslashes in double-quoted strings, note that all backslashes in a regex expression must be escaped like "\\w*".

Let’s see how this works with the following log line and stage. Since source is omitted, the replacement occurs on the log line itself.

2023-01-01T01:00:00.000000001Z stderr P i'm a log message who has sensitive information with password xyz!

stage.replace {
    expression = "password (\\S+)"
    replace    = "*****"
}

The log line is transformed to

2023-01-01T01:00:00.000000001Z stderr P i'm a log message who has sensitive information with password *****!

If replace is empty, then the captured value is omitted instead.

In the following example, source is defined.

{"time":"2023-01-01T01:00:00.000000001Z", "level": "info", "msg":"11.11.11.11 - \"POST /loki/api/push/ HTTP/1.1\" 200 932 \"-\" \"Mozilla/5.0\"}

stage.json {
    expressions = { "level" = "", "msg" = "" }
}

stage.replace {
    expression = "\\S+ - \"POST (\\S+) .*"
    source     = "msg"
    replace    = "redacted_url"
}

The JSON stage adds the following key-value pairs into the extracted map:

time: 2023-01-01T01:00:00.000000001Z
level: info
msg: "11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0"

The replace stage acts on the msg value. The capture group matches against /loki/api/push and is replaced by redacted_url.

The msg value is finally transformed into:

msg: "11.11.11.11 - "POST redacted_url HTTP/1.1" 200 932 "-" "Mozilla/5.0"

The replace field can use a set of templating functions, by utilizing Go’s text/template package.

Let’s see how this works with named capture groups with a sample log line and stage.

11.11.11.11 - agent [01/Jan/2023:00:00:01 +0200]

stage.replace {
    expression = "^(?P<ip>\\S+) (?P<identd>\\S+) (?P<user>\\S+) \\[(?P<timestamp>[\\w:/]+\\s[+\\-]\\d{4})\\]"
    replace    = "{{ .Value | ToUpper }}"
}

Since source is empty, the regex parses the log line itself and extracts the named capture groups to the shared map of values. The replace field acts on these extracted values and converts them to uppercase:

ip: 11.11.11.11
identd: -
user: FRANK
timestamp: 01/JAN/2023:00:00:01 +0200

and the log line becomes:

11.11.11.11 - FRANK [01/JAN/2023:00:00:01 +0200]

The following list contains available functions with examples of more complex replace fields.

ToLower, ToUpper, Replace, Trim, TrimLeftTrimRight, TrimPrefix, TrimSuffix, TrimSpace, Hash, Sha2Hash, regexReplaceAll, regexReplaceAllLiteral

"{{ if eq .Value \"200\" }}{{ Replace .Value \"200\" \"HttpStatusOk\" -1 }}{{ else }}{{ .Value | ToUpper }}{{ end }}"
"*IP4*{{ .Value | Hash "salt" }}*"

stage.sampling block

The sampling stage is used to sample the logs. Configuring the value rate = 0.1 means that 10% of the logs will continue to be processed. The remaining 90% of the logs will be dropped.

The following arguments are supported:

Name	Type	Description	Default	Required
`rate`	`float`	The sampling rate in a range of `[0, 1]`		yes
`drop_counter_reason`	`string`	The label to add to `loki_process_dropped_lines_total` metric when logs are dropped by this stage.	sampling_stage	no

For example, the configuration below will sample 25% of the logs and drop the remaining 75%. When logs are dropped, the loki_process_dropped_lines_total metric is incremented with an additional reason=logs_sampling label.

stage.sampling {
    rate = 0.25
    drop_counter_reason = "logs_sampling"
}

stage.static_labels block

The stage.static_labels inner block configures a static_labels processing stage that adds a static set of labels to incoming log entries.

The following arguments are supported:

Name	Type	Description	Default	Required
`values`	`map(string)`	Configures a `static_labels` processing stage.	`{}`	no

stage.static_labels {
    values = {
      foo = "fooval",
      bar = "barval",
    }
}

stage.template block

The stage.template inner block configures a transforming stage that allows users to manipulate the values in the extracted map by using Go’s text/template package syntax. This stage is primarily useful for manipulating and standardizing data from previous stages before setting them as labels in a subsequent stage. Example use cases are replacing spaces with underscores, converting uppercase strings to lowercase, or hashing a value.

The template stage can also create new keys in the extracted map.

The following arguments are supported:

Name	Type	Description	Default	Required
`source`	`string`	Name from extracted data to parse. If the key doesn’t exist, a new entry is created.		yes
`template`	`string`	Go template string to use.		yes

The template string can be any valid template that can be used by Go’s text/template. It supports all functions from the sprig package, as well as the following list of custom functions:

ToLower, ToUpper, Replace, Trim, TrimLeftTrimRight, TrimPrefix, TrimSuffix, TrimSpace, Hash, Sha2Hash, regexReplaceAll, regexReplaceAllLiteral

More details on each of these functions can be found in the supported functions section below.

Assuming no data is present on the extracted map, the following stage simply adds the new_key: "hello_world"key-value pair to the shared map.

stage.template {
    source   = "new_key"
    template = "hello_world"
}

If the source value exists in the extract fields, its value can be referred to as .Value in the template. The next stage takes the current value of app from the extracted map, converts it to lowercase, and adds a suffix to its value:

stage.template {
    source   = "app"
    template = "{{ ToLower .Value }}_some_suffix"
}

Any previously extracted keys are available for template to expand and use. The next stage takes the current values for level, app and module and creates a new key named output_message:

stage.template {
    source   = "output_msg"
    template = "{{ .level }} for app {{ ToUpper .app }} in module {{.module}}"
}

A special key named Entry can be used to reference the current line; this can be useful when you need to append/prepend something to the log line, like this snippet:

stage.template {
    source   = "message"
    template = "{{.app }}: {{ .Entry }}"
}
stage.output {
    source = "message"
}

Supported functions

In addition to supporting all functions from the sprig package, the template stage supports the following custom functions.

ToLower and ToUpper

ToLower and ToUpper convert the entire string to lowercase and uppercase, respectively.

Examples:

stage.template {
    source   = "out"
    template = "{{ ToLower .app }}"
}
stage.template {
    source   = "out"
    template = "{{ .app | ToUpper }}"
}

Replace

The Replace function syntax is defined as {{ Replace <string> <old> <new> <n> }}.

The function returns a copy of the input string, with instances of the <old> argument being replaced by <new>. The function replaces up to <n> non-overlapping instances of the second argument. If <n> is less than zero, there is no limit on the number of replacement. Finally, if <old> is empty, it matches before and after every UTF-8 character in the string.

This example replaces the first two instances of the loki word by Loki:

stage.template {
    source   = "output"
    template = "{{ Replace .Value "loki" "Loki" 2 }}"
}

Trim, TrimLeft, TrimRight, TrimSpace, TrimPrefix, TrimSuffix

Trim returns a slice of the string s with all leading and trailing Unicode code points contained in cutset removed.
TrimLeft and TrimRight are the same as Trim except that they trim only leading and trailing characters, respectively.
TrimSpace returns a slice of the string s, with all leading and trailing white space removed, as defined by Unicode.
TrimPrefix and TrimSuffix trim the supplied prefix or suffix, respectively. Examples:

stage.template {
    source   = "output"
    template = "{{ Trim .Value ",. " }}"
}
stage.template {
    source   = "output"
    template = "{{ TrimSpace .Value }}"
}
stage.template {
    source   = "output"
    template = "{{ TrimPrefix .Value "--" }}"
}

Regex

regexReplaceAll returns a copy of the input string, replacing matches of the Regexp with the replacement string. Inside the replacement string, $ characters are interpreted as in Expand functions, so for instance, $1 represents the first captured submatch.

regexReplaceAllLiteral returns a copy of the input string, replacing matches of the Regexp with the replacement string. The replacement string is substituted directly, without using Expand.

stage.template {
    source   = "output"
    template = "{{ regexReplaceAll "(a*)bc" .Value "${1}a" }}"
}
stage.template {
    source   = "output"
    template = "{{ regexReplaceAllLiteral "(ts=)" .Value "timestamp=" }}"
}

Hash and Sha2Hash

Hash returns a Sha3_256 hash of the string, represented as a hexadecimal number of 64 digits. You can use it to obfuscate sensitive data and PII in the logs. It requires a (fixed) salt value, to add complexity to low input domains (e.g., all possible social security numbers). Sha2Hash returns a Sha2_256 of the string which is faster and less CPU-intensive than Hash, however it is less secure.

Examples:

stage.template {
    source   = "output"
    template = "{{ Hash .Value "salt" }}"
}
stage.template {
    source   = "output"
    template = "{{ Sha2Hash .Value "salt" }}"
}

We recommend using Hash as it has a stronger hashing algorithm.

stage.tenant block

The stage.tenant inner block sets the tenant ID for the log entry by obtaining it from a field in the extracted data map, a label, or a provided value.

The following arguments are supported:

Name	Type	Description	Default	Required
`label`	`string`	The label to set as tenant ID.	`""`	no
`source`	`string`	The name from the extracted value to use as tenant ID.	`""`	no
`value`	`string`	The value to set as the tenant ID.	`""`	no

The block expects only one of label, source or value to be provided.

The following stage assigns the fixed value team-a as the tenant ID:

stage.tenant {
    value = "team-a"
}

This stage extracts the tenant ID from the customer_id field after parsing the log entry as JSON in the shared extracted map:

stage.json {
    expressions = { "customer_id" = "" }
}
stage.tenant {
    source = "customer_id"
}

The final example extracts the tenant ID from a label set by a previous stage:

stage.labels {
    "namespace" = "k8s_namespace"
}
stage.tenant {
    label = "namespace"
}

stage.timestamp block

The stage.timestamp inner block configures a processing stage that sets the timestamp of log entries before they’re forwarded to the next component. When no timestamp stage is set, the log entry timestamp defaults to the time when the log entry was scraped.

The following arguments are supported:

Name	Type	Description	Default	Required
`source`	`string`	Name from extracted values map to use for the timestamp.		yes
`format`	`string`	Determines how to parse the source string.		yes
`fallback_formats`	`list(string)`	Fallback formats to try if the `format` field fails.	`[]`	no
`location`	`string`	IANA Timezone Database location to use when parsing.	`""`	no
`action_on_failure`	`string`	What to do when the timestamp can’t be extracted or parsed.	`"fudge"`	no

Note
Be careful with further stages which may also override the timestamp. For example, a stage.pack with ingest_timestamp set to true could replace the timestamp which stage.timestamp had set earlier in the pipeline.

The source field defines which value from the shared map of extracted values the stage should attempt to parse as a timestamp.

The format field defines how that source should be parsed.

First off, the format can be set to one of the following shorthand values for commonly-used forms:

ANSIC: Mon Jan _2 15:04:05 2006
UnixDate: Mon Jan _2 15:04:05 MST 2006
RubyDate: Mon Jan 02 15:04:05 -0700 2006
RFC822: 02 Jan 06 15:04 MST
RFC822Z: 02 Jan 06 15:04 -0700
RFC850: Monday, 02-Jan-06 15:04:05 MST
RFC1123: Mon, 02 Jan 2006 15:04:05 MST
RFC1123Z: Mon, 02 Jan 2006 15:04:05 -0700
RFC3339: 2006-01-02T15:04:05-07:00
RFC3339Nano: 2006-01-02T15:04:05.999999999-07:00

Additionally, support for common Unix timestamps is supported with the following format values:

Unix: 1562708916 or with fractions 1562708916.000000123
UnixMs: 1562708916414
UnixUs: 1562708916414123
UnixNs: 1562708916000000123

Otherwise, the field accepts a custom format string that defines how an arbitrary reference point in history should be interpreted by the stage. The arbitrary reference point is Mon Jan 2 15:04:05 -0700 MST 2006.

The string value of the field is passed directly to the layout parameter in Go’s time.Parse function.

If the custom format has no year component, the stage uses the current year, according to the system’s clock.

The following table shows the supported reference values to use when defining a custom format.

Timestamp Component	Format value
Year	06, 2006
Month	1, 01, Jan, January
Day	2, 02, _2 (two digits right justified)
Day of the week	Mon, Monday
Hour	3 (12-hour), 03 (12-hour zero prefixed), 15 (24-hour)
Minute	4, 04
Second	5, 05
Fraction of second	.000 (ms zero prefixed), .000000 (μs), .000000000 (ns), .999 (ms without trailing zeroes), .999999 (μs), .999999999 (ns)
12-hour period	pm, PM
Timezone name	MST
Timezone offset	-0700, -070000 (with seconds), -07, 07:00, -07:00:00 (with seconds)
Timezone ISO-8601	Z0700 (Z for UTC or time offset), Z070000, Z07, Z07:00, Z07:00:00

The fallback_formats field defines one or more format fields to try and parse the timestamp with, if parsing with format fails.

The location field must be a valid IANA Timezone Database location and determines in which timezone the timestamp value is interpreted to be in.

The action_on_failure field defines what should happen when the source field doesn’t exist in the shared extracted map, or if the timestamp parsing fails.

The supported actions are:

fudge (default): Change the timestamp to the last known timestamp, summing up 1 nanosecond (to guarantee log entries ordering).
skip: Do not change the timestamp and keep the time when the log entry was scraped.

The following stage fetches the time value from the shared values map, parses it as a RFC3339 format, and sets it as the log entry’s timestamp.

stage.timestamp {
    source = "time"
    format = "RFC3339"
}

stage.geoip block

The stage.geoip inner block configures a processing stage that reads an IP address and populates the shared map with geoip fields. Maxmind’s GeoIP2 database is used for the lookup.

The following arguments are supported:

Name	Type	Description	Required
`db`	`string`	Path to the Maxmind DB file.	yes
`source`	`string`	IP from extracted data to parse.	yes
`db_type`	`string`	Maxmind DB type. Allowed values are “city”, “asn”, “country”.	no
`custom_lookups`	`map(string)`	Key-value pairs of JMESPath expressions.	no

GeoIP with City database example:

{"log":"log message","client_ip":"34.120.177.193"}

loki.process "example" {
	stage.json {
		expressions = {ip = "client_ip"}
	}

	stage.geoip {
		source  = "ip"
		db      = "/path/to/db/GeoLite2-City.mmdb"
		db_type = "city"
	}

	stage.labels {
		values = {
			geoip_city_name          = "",
			geoip_country_name       = "",
			geoip_country_code       = "",
			geoip_continent_name     = "",
			geoip_continent_code     = "",
			geoip_location_latitude  = "",
			geoip_location_longitude = "",
			geoip_postal_code        = "",
			geoip_timezone           = "",
			geoip_subdivision_name   = "",
			geoip_subdivision_code   = "",
		}
	}
}

The json stage extracts the IP address from the client_ip key in the log line. Then the extracted ip value is given as source to geoip stage. The geoip stage performs a lookup on the IP and populates the following fields in the shared map which are added as labels using the labels stage.

The extracted data from the IP used in this example:

geoip_city_name: Kansas City
geoip_country_name: United States
geoip_country_code: US
geoip_continent_name: North America
geoip_continent_code: NA
geoip_location_latitude: 39.1027
geoip_location_longitude: -94.5778
geoip_postal_code: 64184
geoip_timezone: America/Chicago
geoip_subdivision_name: Missouri
geoip_subdivision_code: MO

GeoIP with ASN (Autonomous System Number) database example

loki.process "example" {
	stage.json {
		expressions = {ip = "client_ip"}
	}

	stage.geoip {
		source  = "ip"
		db      = "/path/to/db/GeoIP2-ASN.mmdb"
		db_type = "asn"
	}

	stage.labels {
		values = {
			geoip_autonomous_system_number       = "",
			geoip_autonomous_system_organization = "",
		}
	}
}

The extracted data from the IP used in this example:

geoip_autonomous_system_number: 396982
geoip_autonomous_system_organization: GOOGLE-CLOUD-PLATFORM

GeoIP with Country database example:

{"log":"log message","client_ip":"34.120.177.193"}

loki.process "example" {
	stage.json {
		expressions = {ip = "client_ip"}
	}

	stage.geoip {
		source  = "ip"
		db      = "/path/to/db/GeoLite2-Country.mmdb"
		db_type = "country"
	}

	stage.labels {
		values = {
			geoip_country_name       = "",
			geoip_country_code       = "",
			geoip_continent_name     = "",
			geoip_continent_code     = "",
		}
	}
}

The extracted data from the IP used in this example:

geoip_country_name: United States
geoip_country_code: US
geoip_continent_name: North America
geoip_continent_code: NA

GeoIP with custom fields example

If the MMDB file used is enriched with custom data, for example, private IP addresses as explained in the Maxmind blog post, then it can be extracted from the record using the custom_lookups attribute.

loki.process "example" {
	stage.json {
		expressions = {ip = "client_ip"}
	}

	stage.geoip {
		source         = "ip"
		db             = "/path/to/db/GeoIP2-Enriched.mmdb"
		db_type        = "city"
		custom_lookups = {
			"department"  = "MyCompany.DeptName",
			"parent_vnet" = "MyCompany.ParentVNet",
			"subnet"      = "MyCompany.Subnet",
		}
	}

	stage.labels {
		values = {
			department  = "",
			parent_vnet = "",
			subnet      = "",
		}
	}
}

The json stage extracts the IP address from the client_ip key in the log line. Then the extracted ip value is given as source to geoip stage. The geoip stage performs a lookup on the IP and populates the shared map with the data from the city database results in addition to the custom lookups. Lastly, the custom lookup fields from the shared map are added as labels.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`LogsReceiver`	A value that other components can use to send log entries to.

Component health

loki.process is only reported as unhealthy if given an invalid configuration.

Debug information

loki.process does not expose any component-specific debug information.

Debug metrics

loki_process_dropped_lines_total (counter): Number of lines dropped as part of a processing stage.
loki_process_dropped_lines_by_label_total (counter): Number of lines dropped when by_label_name is non-empty in stage.limit.

Example

This example creates a loki.process component that extracts the environment value from a JSON log line and sets it as a label named ’env’.

loki.process "local" {
  forward_to = [loki.write.onprem.receiver]

  stage.json {
      expressions = { "extracted_env" = "environment" }
  }

  stage.labels {
      values = { "env" = "extracted_env" }
  }
}

Compatible components

loki.process can accept arguments from the following components:

Components that export Loki LogsReceiver

loki.process has exports that can be consumed by the following components:

Components that consume Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.relabel

Fri, 10 Apr 2026 07:58:08 +0000

loki.relabel

The loki.relabel component rewrites the label set of each log entry passed to its receiver by applying one or more relabeling rules and forwards the results to the list of receivers in the component’s arguments.

If no labels remain after the relabeling rules are applied, then the log entries are dropped.

The most common use of loki.relabel is to filter log entries or standardize the label set that is passed to one or more downstream receivers. The rule blocks are applied to the label set of each log entry in order of their appearance in the configuration file. The configured rules can be retrieved by calling the function in the rules export field.

If you’re looking for a way to process the log entry contents, take a look at the loki.process component instead.

Multiple loki.relabel components can be specified by giving them different labels.

Usage

loki.relabel "LABEL" {
  forward_to = RECEIVER_LIST

  rule {
    ...
  }

  ...
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`forward_to`	`list(receiver)`	Where to forward log entries after relabeling.		yes
`max_cache_size`	`int`	The maximum number of elements to hold in the relabeling cache	10,000	no

Blocks

The following blocks are supported inside the definition of loki.relabel:

Hierarchy	Name	Description	Required
rule	rule	Relabeling rules to apply to received log entries.	no

rule block

The rule block contains the definition of any relabeling rules that can be applied to an input log. The transformations are applied in top-down order if more than one rule block is defined.

You can use the following arguments to configure a rule. All arguments are optional. Omitted fields take their default values.

Name	Type	Description	Default	Required
`action`	`string`	The relabeling action to perform.	replace	no
`modulus`	`uint`	A positive integer used to calculate the modulus of the hashed source label values.		no
`regex`	`string`	A valid RE2 expression with support for parenthesized capture groups. Used to match the extracted value from the combination of the `source_label` and `separator` fields or filter labels during the `labelkeep/labeldrop/labelmap` actions.	`(.*)`	no
`replacement`	`string`	The value against which a regular expression replace is performed if the regular expression matches the extracted value. Supports previously captured groups.	`"$1"`	no
`separator`	`string`	The separator used to concatenate the values present in `source_labels`.	;	no
`source_labels`	`list(string)`	The list of labels whose values are to be selected. Their content is concatenated using the `separator` and matched against `regex`.		no
`target_label`	`string`	Label to which the resulting value is written to.		no

You can use the following actions:

drop - Drops logs where regex matches the string extracted using the source_labels and separator.
dropequal - Drop targets for which the concatenated source_labels do match target_label.
hashmod - Hashes the concatenated labels, calculates its modulo modulus, and writes the result to the target_label.
keep - Keeps logs where regex matches the string extracted using the source_labels and separator.
keepequal - Drop targets for which the concatenated source_labels do not match target_label.
labeldrop - Matches regex against all label names. Any labels that match are removed from the log’s label set.
labelkeep - Matches regex against all label names. Any labels that don’t match are removed from the log’s label set.
labelmap - Matches regex against all label names. Any labels that match are renamed according to the contents of the replacement field.
lowercase - Sets target_label to the lowercase form of the concatenated source_labels.
replace - Matches regex to the concatenated labels. If there’s a match, it replaces the content of the target_label using the contents of the replacement field.
uppercase - Sets target_label to the uppercase form of the concatenated source_labels.

Note
The regular expression capture groups can be referred to using either the $CAPTURE_GROUP_NUMBER or ${CAPTURE_GROUP_NUMBER} notation.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`receiver`	The input receiver where log lines are sent to be relabeled.
`rules`	`RelabelRules`	The currently configured relabeling rules.

Component health

loki.relabel is only reported as unhealthy if given an invalid configuration. In those cases, exported fields are kept at their last healthy values.

Debug information

loki.relabel does not expose any component-specific debug information.

Debug metrics

loki_relabel_entries_processed (counter): Total number of log entries processed.
loki_relabel_entries_written (counter): Total number of log entries forwarded.
loki_relabel_cache_misses (counter): Total number of cache misses.
loki_relabel_cache_hits (counter): Total number of cache hits.
loki_relabel_cache_size (gauge): Total size of relabel cache.

Example

The following example creates a loki.relabel component that only forwards entries whose ’level’ value is set to ’error'.

loki.relabel "keep_error_only" {
  forward_to = [loki.write.onprem.receiver]

  rule {
    action        = "keep"
    source_labels = ["level"]
    regex         = "error"
  }
}

Compatible components

loki.relabel can accept arguments from the following components:

Components that export Loki LogsReceiver

loki.relabel has exports that can be consumed by the following components:

Components that consume Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.rules.kubernetes

Fri, 10 Apr 2026 07:58:08 +0000

loki.rules.kubernetes

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

loki.rules.kubernetes discovers PrometheusRule Kubernetes resources and loads them into a Loki instance.

You can specify multiple loki.rules.kubernetes components by giving them different labels.
Kubernetes label selectors can be used to limit the Namespace and PrometheusRule resources considered during reconciliation.
Compatible with the Ruler APIs of Grafana Loki, Grafana Cloud, and Grafana Enterprise Metrics.
Compatible with the PrometheusRule CRD from the prometheus-operator.
This component accesses the Kubernetes REST API from within a Pod.

Note
This component requires [Role-based access control (RBAC)][] to be set up in Kubernetes for Grafana Agent to access it via the Kubernetes REST API.

Role-based access control (RBAC)]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/

Usage

loki.rules.kubernetes "LABEL" {
  address = LOKI_RULER_URL
}

Arguments

loki.rules.kubernetes supports the following arguments:

Name	Type	Description	Default	Required
`address`	`string`	URL of the Loki ruler.		yes
`tenant_id`	`string`	Loki tenant ID.		no
`use_legacy_routes`	`bool`	Whether to use deprecated ruler API endpoints.	false	no
`sync_interval`	`duration`	Amount of time between reconciliations with Loki.	“30s”	no
`loki_namespace_prefix`	`string`	Prefix used to differentiate multiple Grafana Agent deployments.	“agent”	no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`proxy_url`	`string`	HTTP proxy to proxy requests through.		no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no

At most, one of the following can be provided:

If no tenant_id is provided, the component assumes that the Loki instance at address is running in single-tenant mode and no X-Scope-OrgID header is sent.

The sync_interval argument determines how often Loki’s ruler API is accessed to reload the current state. Interaction with the Kubernetes API works differently. Updates are processed as events from the Kubernetes API server according to the informer pattern.

You can use the loki_namespace_prefix argument to separate the rules managed by multiple Grafana Agent deployments across your infrastructure. You should set the prefix to a unique value for each deployment.

Blocks

The following blocks are supported inside the definition of loki.rules.kubernetes:

Hierarchy	Block	Description	Required
rule_namespace_selector	label_selector	Label selector for `Namespace` resources.	no
rule_namespace_selector > match_expression	match_expression	Label match expression for `Namespace` resources.	no
rule_selector	label_selector	Label selector for `PrometheusRule` resources.	no
rule_selector > match_expression	match_expression	Label match expression for `PrometheusRule` resources.	no
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

label_selector block

The label_selector block describes a Kubernetes label selector for rule or namespace discovery.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_labels`	`map(string)`	Label keys and values used to discover resources.	`{}`	yes

When the match_labels argument is empty, all resources will be matched.

match_expression block

The match_expression block describes a Kubernetes label match expression for rule or namespace discovery.

The following arguments are supported:

Name	Type	Description	Required
`key`	`string`	The label name to match against.	yes
`operator`	`string`	The operator to use when matching.	yes
`values`	`list(string)`	The values used when matching.	no

The operator argument should be one of the following strings:

"in"
"notin"
"exists"

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

loki.rules.kubernetes does not export any fields.

Component health

loki.rules.kubernetes is reported as unhealthy if given an invalid configuration or an error occurs during reconciliation.

Debug information

loki.rules.kubernetes exposes resource-level debug information.

The following are exposed per discovered PrometheusRule resource:

The Kubernetes namespace.
The resource name.
The resource uid.
The number of rule groups.

The following are exposed per discovered Loki rule namespace resource:

The namespace name.
The number of rule groups.

Only resources managed by the component are exposed - regardless of how many actually exist.

Debug metrics

Metric Name	Type	Description
`loki_rules_config_updates_total`	`counter`	Number of times the configuration has been updated.
`loki_rules_events_total`	`counter`	Number of events processed, partitioned by event type.
`loki_rules_events_failed_total`	`counter`	Number of events that failed to be processed, partitioned by event type.
`loki_rules_events_retried_total`	`counter`	Number of events that were retried, partitioned by event type.
`loki_rules_client_request_duration_seconds`	`histogram`	Duration of requests to the Loki API.

Example

This example creates a loki.rules.kubernetes component that loads discovered rules to a local Loki instance under the team-a tenant. Only namespaces and rules with the agent label set to yes are included.

loki.rules.kubernetes "local" {
    address = "loki:3100"
    tenant_id = "team-a"

    rule_namespace_selector {
        match_labels = {
            agent = "yes",
        }
    }

    rule_selector {
        match_labels = {
            agent = "yes",
        }
    }
}

This example creates a loki.rules.kubernetes component that loads discovered rules to Grafana Cloud.

loki.rules.kubernetes "default" {
    address = "GRAFANA_CLOUD_URL"
    basic_auth {
        username = "GRAFANA_CLOUD_USER"
        password = "GRAFANA_CLOUD_API_KEY"
        // Alternatively, load the password from a file:
        // password_file = "GRAFANA_CLOUD_API_KEY_PATH"
    }
}

The following example is an RBAC configuration for Kubernetes. It authorizes Grafana Agent to query the Kubernetes REST API:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: grafana-agent
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: grafana-agent
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["monitoring.coreos.com"]
  resources: ["prometheusrules"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: grafana-agent
subjects:
- kind: ServiceAccount
  name: grafana-agent
  namespace: default
roleRef:
  kind: ClusterRole
  name: grafana-agent
  apiGroup: rbac.authorization.k8s.io

loki.source.api

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.api

loki.source.api receives log entries over HTTP and forwards them to other loki.* components.

The HTTP API exposed is compatible with Loki push API and the logproto format. This means that other loki.write components can be used as a client and send requests to loki.source.api which enables using the Agent as a proxy for logs.

Usage

loki.source.api "LABEL" {
    http {
        listen_address = "LISTEN_ADDRESS"
        listen_port = PORT
    }
    forward_to = RECEIVER_LIST
}

The component will start HTTP server on the configured port and address with the following endpoints:

/loki/api/v1/push - accepting POST requests compatible with Loki push API, for example, from another Grafana Agent’s loki.write component.
/loki/api/v1/raw - accepting POST requests with newline-delimited log lines in body. This can be used to send NDJSON or plaintext logs. This is compatible with promtail’s push API endpoint - see promtail’s documentation for more information. NOTE: when this endpoint is used, the incoming timestamps cannot be used and the use_incoming_timestamp = true setting will be ignored.
/loki/ready - accepting GET requests - can be used to confirm the server is reachable and healthy.
/api/v1/push - internally reroutes to /loki/api/v1/push
/api/v1/raw - internally reroutes to /loki/api/v1/raw

Arguments

loki.source.api supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`use_incoming_timestamp`	`bool`	Whether or not to use the timestamp received from request.	`false`	no
`labels`	`map(string)`	The labels to associate with each received logs record.	`{}`	no
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no

The relabel_rules field can make use of the rules export value from a loki.relabel component to apply one or more relabeling rules to log entries before they’re forwarded to the list of receivers in forward_to.

Blocks

The following blocks are supported inside the definition of loki.source.api:

Hierarchy	Name	Description	Required
`http`	http	Configures the HTTP server that receives requests.	no

http

The http block configures the HTTP server.

You can use the following arguments to configure the http block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. Defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections.	`8080`	no
`server_idle_timeout`	`duration`	Idle timeout for HTTP server.	`"120s"`	no
`server_read_timeout`	`duration`	Read timeout for HTTP server.	`"30s"`	no
`server_write_timeout`	`duration`	Write timeout for HTTP server.	`"30s"`	no

Exported fields

loki.source.api does not export any fields.

Component health

loki.source.api is only reported as unhealthy if given an invalid configuration.

Debug metrics

The following are some of the metrics that are exposed when this component is used. Note that the metrics include labels such as status_code where relevant, which can be used to measure request success rates.

loki_source_api_request_duration_seconds (histogram): Time (in seconds) spent serving HTTP requests.
loki_source_api_request_message_bytes (histogram): Size (in bytes) of messages received in the request.
loki_source_api_response_message_bytes (histogram): Size (in bytes) of messages sent in response.
loki_source_api_tcp_connections (gauge): Current number of accepted TCP connections.

Example

This example starts an HTTP server on 0.0.0.0 address and port 9999. The server receives log entries and forwards them to a loki.write component while adding a forwarded="true" label. The loki.write component will send the logs to the specified loki instance using basic auth credentials provided.

loki.write "local" {
    endpoint {
        url = "http://loki:3100/api/v1/push"
        basic_auth {
            username = "<your username>"
            password_file = "<your password file>"
        }
    }
}

loki.source.api "loki_push_api" {
    http {
        listen_address = "0.0.0.0"
        listen_port = 9999
    }
    forward_to = [
        loki.write.local.receiver,
    ]
    labels = {
        forwarded = "true",
    }
}

Compatible components

loki.source.api can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.awsfirehose

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.awsfirehose

loki.source.awsfirehose receives log entries over HTTP from AWS Firehose and forwards them to other loki.* components.

The HTTP API exposed is compatible with the Firehose HTTP Delivery API. Since the API model that AWS Firehose uses to deliver data over HTTP is generic enough, the same component can be used to receive data from multiple origins:

The component uses a heuristic to try to decode as much information as possible from each log record, and it falls back to writing the raw records to Loki. The decoding process goes as follows:

AWS Firehose sends batched requests
Each record is treated individually
For each record received in each request:
- If the record comes from a CloudWatch logs subscription filter, it is decoded and each logging event is written to Loki
- All other records are written raw to Loki

The component exposes some internal labels, available for relabeling. The following tables describes internal labels available in records coming from any source.

Name	Description	Example
`__aws_firehose_request_id`	Firehose request ID.	`a1af4300-6c09-4916-ba8f-12f336176246`
`__aws_firehose_source_arn`	Firehose delivery stream ARN.	`arn:aws:firehose:us-east-2:123:deliverystream/aws_firehose_test_stream`

If the source of the Firehose record is CloudWatch logs, the request is further decoded and enriched with even more labels, exposed as follows:

Name	Description	Example
`__aws_owner`	The AWS Account ID of the originating log data.	`111111111111`
`__aws_cw_log_group`	The log group name of the originating log data.	`CloudTrail/logs`
`__aws_cw_log_stream`	The log stream name of the originating log data.	`111111111111_CloudTrail/logs_us-east-1`
`__aws_cw_matched_filters`	The list of subscription filter names that match the originating log data. The list is encoded as a comma-separated list.	`Destination,Destination2`
`__aws_cw_msg_type`	Data messages will use the `DATA_MESSAGE` type. Sometimes CloudWatch Logs may emit Kinesis Data Streams records with a `CONTROL_MESSAGE` type, mainly for checking if the destination is reachable.	`DATA_MESSAGE`

See Examples for a full example configuration showing how to enrich each log entry with these labels.

Usage

loki.source.awsfirehose "LABEL" {
    http {
        listen_address = "LISTEN_ADDRESS"
        listen_port = PORT 
    }
    forward_to = RECEIVER_LIST
}

The component will start an HTTP server on the configured port and address with the following endpoints:

/awsfirehose/api/v1/push - accepting POST requests compatible with AWS Firehose HTTP Specifications.

Arguments

loki.source.awsfirehose supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`use_incoming_timestamp`	`bool`	Whether or not to use the timestamp received from the request.	`false`	no
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no
`access_key`	`secret`	If set, require AWS Firehose to provide a matching key.	`""`	no

Blocks

The following blocks are supported inside the definition of loki.source.awsfirehose:

Hierarchy	Name	Description	Required
`http`	http	Configures the HTTP server that receives requests.	no
`grpc`	grpc	Configures the gRPC server that receives requests.	no

http

The http block configures the HTTP server.

You can use the following arguments to configure the http block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. Defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections.	`8080`	no
`server_idle_timeout`	`duration`	Idle timeout for HTTP server.	`"120s"`	no
`server_read_timeout`	`duration`	Read timeout for HTTP server.	`"30s"`	no
`server_write_timeout`	`duration`	Write timeout for HTTP server.	`"30s"`	no

grpc

The grpc block configures the gRPC server.

You can use the following arguments to configure the grpc block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. It defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections. Defaults to a random free port.	`0`	no
`max_connection_age_grace`	`duration`	An additive period after `max_connection_age` after which the connection is forcibly closed.	`"infinity"`	no
`max_connection_age`	`duration`	The duration for the maximum time a connection may exist before it is closed.	`"infinity"`	no
`max_connection_idle`	`duration`	The duration after which an idle connection is closed.	`"infinity"`	no
`server_max_concurrent_streams`	`int`	Limit on the number of concurrent streams for gRPC calls (0 = unlimited).	`100`	no
`server_max_recv_msg_size`	`int`	Limit on the size of a gRPC message this server can receive (bytes).	`4MB`	no
`server_max_send_msg_size`	`int`	Limit on the size of a gRPC message this server can send (bytes).	`4MB`	no

Exported fields

loki.source.awsfirehose does not export any fields.

Component health

loki.source.awsfirehose is only reported as unhealthy if given an invalid configuration.

Debug metrics

The following are some of the metrics that are exposed when this component is used.

Note
The metrics include labels such as status_code where relevant, which you can use to measure request success rates.

loki_source_awsfirehose_request_errors (counter): Count of errors while receiving a request.
loki_source_awsfirehose_record_errors (counter): Count of errors while decoding an individual record.
loki_source_awsfirehose_records_received (counter): Count of records received.
loki_source_awsfirehose_batch_size (histogram): Size (in units) of the number of records received per request.

Example

This example starts an HTTP server on 0.0.0.0 address and port 9999. The server receives log entries and forwards them to a loki.write component. The loki.write component will send the logs to the specified loki instance using basic auth credentials provided.

loki.write "local" {
    endpoint {
        url = "http://loki:3100/api/v1/push"
        basic_auth {
            username = "<your username>"
            password_file = "<your password file>"
        }
    }
}

loki.source.awsfirehose "loki_fh_receiver" {
    http {
        listen_address = "0.0.0.0"
        listen_port = 9999
    }
    forward_to = [
        loki.write.local.receiver,
    ]
}

As another example, if you are receiving records that originated from a CloudWatch logs subscription, you can enrich each received entry by relabeling internal labels. The following configuration builds upon the one above but keeps the origin log stream and group as log_stream and log_group, respectively.

loki.write "local" {
    endpoint {
        url = "http://loki:3100/api/v1/push"
        basic_auth {
            username = "<your username>"
            password_file = "<your password file>"
        }
    }
}

loki.source.awsfirehose "loki_fh_receiver" {
    http {
        listen_address = "0.0.0.0"
        listen_port = 9999
    }
    forward_to = [
        loki.write.local.receiver,
    ]
    relabel_rules = loki.relabel.logging_origin.rules
}

loki.relabel "logging_origin" {
  rule {
    action = "replace"
    source_labels = ["__aws_cw_log_group"]
    target_label = "log_group"
  }
  rule {
    action = "replace"
    source_labels = ["__aws_cw_log_stream"]
    target_label = "log_stream"
  }
  forward_to = []
}

Compatible components

loki.source.awsfirehose can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.azure_event_hubs

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.azure_event_hubs

loki.source.azure_event_hubs receives Azure Event Hubs messages by making use of an Apache Kafka endpoint on Event Hubs. For more information, see the Azure Event Hubs documentation.

To learn more about streaming Azure logs to an Azure Event Hubs, refer to Microsoft’s tutorial on how to Stream Azure Active Directory logs to an Azure event hub.

Note that an Apache Kafka endpoint is not available within the Basic pricing plan. For more information, see the Event Hubs pricing page.

Multiple loki.source.azure_event_hubs components can be specified by giving them different labels.

Usage

loki.source.azure_event_hubs "LABEL" {
	fully_qualified_namespace = "HOST:PORT"
	event_hubs                = EVENT_HUB_LIST
	forward_to                = RECEIVER_LIST

	authentication {
		mechanism = "AUTHENTICATION_MECHANISM"
	}
}

Arguments

loki.source.azure_event_hubs supports the following arguments:

Name	Type	Description	Default	Required
`fully_qualified_namespace`	`string`	Event hub namespace.		yes
`event_hubs`	`list(string)`	Event Hubs to consume.		yes
`group_id`	`string`	The Kafka consumer group id.	`"loki.source.azure_event_hubs"`	no
`assignor`	`string`	The consumer group rebalancing strategy to use.	`"range"`	no
`use_incoming_timestamp`	`bool`	Whether or not to use the timestamp received from Azure Event Hub.	`false`	no
`labels`	`map(string)`	The labels to associate with each received event.	`{}`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no
`disallow_custom_messages`	`bool`	Whether to ignore messages that don’t match the schema for Azure resource logs.	`false`	no
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no

The fully_qualified_namespace argument must refer to a full HOST:PORT that points to your event hub, such as NAMESPACE.servicebus.windows.net:9093. The assignor argument must be set to one of "range", "roundrobin", or "sticky".

Labels

The labels map is applied to every message that the component reads.

The following internal labels prefixed with __ are available but are discarded if not relabeled:

__meta_kafka_message_key
__meta_kafka_topic
__meta_kafka_partition
__meta_kafka_member_id
__meta_kafka_group_id
__azure_event_hubs_category

Blocks

The following blocks are supported inside the definition of loki.source.azure_event_hubs:

Hierarchy	Name	Description	Required
authentication	authentication	Authentication configuration with Azure Event Hub.	yes

authentication block

The authentication block defines the authentication method when communicating with Azure Event Hub.

Name	Type	Description	Required
`mechanism`	`string`	Authentication mechanism.	yes
`connection_string`	`string`	Event Hubs ConnectionString for authentication on Azure Cloud.	no
`scopes`	`list(string)`	Access token scopes. Default is `fully_qualified_namespace` without port.	no

mechanism supports the values "connection_string" and "oauth". If "connection_string" is used, you must set the connection_string attribute. If "oauth" is used, you must configure one of the supported credential types as documented here: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/azidentity/README.md#credential-types via environment variables or Azure CLI.

Exported fields

loki.source.azure_event_hubs does not export any fields.

Component health

loki.source.azure_event_hubs is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.azure_event_hubs does not expose additional debug info.

Example

This example consumes messages from Azure Event Hub and uses OAuth to authenticate itself.

loki.source.azure_event_hubs "example" {
	fully_qualified_namespace = "my-ns.servicebus.windows.net:9093"
	event_hubs                = ["gw-logs"]
	forward_to                = [loki.write.example.receiver]

	authentication {
		mechanism = "oauth"
	}
}

loki.write "example" {
	endpoint {
		url = "loki:3100/api/v1/push"
	}
}
```<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`loki.source.azure_event_hubs` can accept arguments from the following components:

- Components that export [Loki `LogsReceiver`](../../compatibility/#loki-logsreceiver-exporters)




Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.


<!-- END GENERATED COMPATIBLE COMPONENTS -->

loki.source.cloudflare

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.cloudflare

loki.source.cloudflare pulls logs from the Cloudflare Logpull API and forwards them to other loki.* components.

These logs contain data related to the connecting client, the request path through the Cloudflare network, and the response from the origin web server and can be useful for enriching existing logs on an origin server.

Multiple loki.source.cloudflare components can be specified by giving them different labels.

Usage

loki.source.cloudflare "LABEL" {
  zone_id   = "ZONE_ID"
  api_token = "API_TOKEN"

  forward_to = RECEIVER_LIST
}

Arguments

loki.source.cloudflare supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`api_token`	`string`	The API token to authenticate with.		yes
`zone_id`	`string`	The Cloudflare zone ID to use.		yes
`labels`	`map(string)`	The labels to associate with incoming log entries.	`{}`	no
`workers`	`int`	The number of workers to use for parsing logs.	`3`	no
`pull_range`	`duration`	The timeframe to fetch for each pull request.	`"1m"`	no
`fields_type`	`string`	The set of fields to fetch for log entries.	`"default"`	no
`additional_fields`	`list(string)`	The additional list of fields to supplement those provided via `fields_type`.		no

By default loki.source.cloudflare fetches logs with the default set of fields. Here are the different sets of fields_type available for selection, and the fields they include:

default includes:

"ClientIP", "ClientRequestHost", "ClientRequestMethod", "ClientRequestURI", "EdgeEndTimestamp", "EdgeResponseBytes", "EdgeRequestHost", "EdgeResponseStatus", "EdgeStartTimestamp", "RayID"

plus any extra fields provided via additional_fields argument.

minimal includes all default fields and adds:

"ZoneID", "ClientSSLProtocol", "ClientRequestProtocol", "ClientRequestPath", "ClientRequestUserAgent", "ClientRequestReferer", "EdgeColoCode", "ClientCountry", "CacheCacheStatus", "CacheResponseStatus", "EdgeResponseContentType"

plus any extra fields provided via additional_fields argument.

extended includes all minimal fields and adds:

"ClientSSLCipher", "ClientASN", "ClientIPClass", "CacheResponseBytes", "EdgePathingOp", "EdgePathingSrc", "EdgePathingStatus", "ParentRayID", "WorkerCPUTime", "WorkerStatus", "WorkerSubrequest", "WorkerSubrequestCount", "OriginIP", "OriginResponseStatus", "OriginSSLProtocol", "OriginResponseHTTPExpires", "OriginResponseHTTPLastModified"

plus any extra fields provided via additional_fields argument.

all includes all extended fields and adds:

 "BotScore", "BotScoreSrc", "BotTags", "ClientRequestBytes", "ClientSrcPort", "ClientXRequestedWith", "CacheTieredFill", "EdgeResponseCompressionRatio", "EdgeServerIP", "FirewallMatchesSources", "FirewallMatchesActions", "FirewallMatchesRuleIDs", "OriginResponseBytes", "OriginResponseTime", "ClientDeviceType", "WAFFlags", "WAFMatchedVar", "EdgeColoID", "RequestHeaders", "ResponseHeaders", "ClientRequestSource"`

plus any extra fields provided via additional_fields argument (this is still relevant in this case if new fields are made available via Cloudflare API but are not yet included in all).

custom includes only the fields defined in additional_fields.

The component saves the last successfully-fetched timestamp in its positions file. If a position is found in the file for a given zone ID, the component restarts pulling logs from that timestamp. When no position is found, the component starts pulling logs from the current time.

Logs are fetched using multiple workers which request the last available pull_range repeatedly. It is possible to fall behind due to having too many log lines to process for each pull; adding more workers, decreasing the pull range, or decreasing the quantity of fields fetched can mitigate this performance issue.

The last timestamp fetched by the component is recorded in the loki_source_cloudflare_target_last_requested_end_timestamp debug metric.

All incoming Cloudflare log entries are in JSON format. You can make use of the loki.process component and a JSON processing stage to extract more labels or change the log line format. A sample log looks like this:

{
    "CacheCacheStatus": "miss",
    "CacheResponseBytes": 8377,
    "CacheResponseStatus": 200,
    "CacheTieredFill": false,
    "ClientASN": 786,
    "ClientCountry": "gb",
    "ClientDeviceType": "desktop",
    "ClientIP": "100.100.5.5",
    "ClientIPClass": "noRecord",
    "ClientRequestBytes": 2691,
    "ClientRequestHost": "www.foo.com",
    "ClientRequestMethod": "GET",
    "ClientRequestPath": "/comments/foo/",
    "ClientRequestProtocol": "HTTP/1.0",
    "ClientRequestReferer": "https://www.foo.com/foo/168855/?offset=8625",
    "ClientRequestURI": "/foo/15248108/",
    "ClientRequestUserAgent": "some bot",
    "ClientRequestSource": "1"
    "ClientSSLCipher": "ECDHE-ECDSA-AES128-GCM-SHA256",
    "ClientSSLProtocol": "TLSv1.2",
    "ClientSrcPort": 39816,
    "ClientXRequestedWith": "",
    "EdgeColoCode": "MAN",
    "EdgeColoID": 341,
    "EdgeEndTimestamp": 1637336610671000000,
    "EdgePathingOp": "wl",
    "EdgePathingSrc": "macro",
    "EdgePathingStatus": "nr",
    "EdgeRateLimitAction": "",
    "EdgeRateLimitID": 0,
    "EdgeRequestHost": "www.foo.com",
    "EdgeResponseBytes": 14878,
    "EdgeResponseCompressionRatio": 1,
    "EdgeResponseContentType": "text/html",
    "EdgeResponseStatus": 200,
    "EdgeServerIP": "8.8.8.8",
    "EdgeStartTimestamp": 1637336610517000000,
    "FirewallMatchesActions": [],
    "FirewallMatchesRuleIDs": [],
    "FirewallMatchesSources": [],
    "OriginIP": "8.8.8.8",
    "OriginResponseBytes": 0,
    "OriginResponseHTTPExpires": "",
    "OriginResponseHTTPLastModified": "",
    "OriginResponseStatus": 200,
    "OriginResponseTime": 123000000,
    "OriginSSLProtocol": "TLSv1.2",
    "ParentRayID": "00",
    "RayID": "6b0a...",
    "RequestHeaders": [],
    "ResponseHeaders": [
      "x-foo": "bar"
    ],
    "SecurityLevel": "med",
    "WAFAction": "unknown",
    "WAFFlags": "0",
    "WAFMatchedVar": "",
    "WAFProfile": "unknown",
    "WAFRuleID": "",
    "WAFRuleMessage": "",
    "WorkerCPUTime": 0,
    "WorkerStatus": "unknown",
    "WorkerSubrequest": false,
    "WorkerSubrequestCount": 0,
    "ZoneID": 1234
}

Exported fields

loki.source.cloudflare does not export any fields.

Component health

loki.source.cloudflare is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.cloudflare exposes the following debug information:

Whether the target is ready and reading logs from the API.
The Cloudflare zone ID.
The last error reported, if any.
The stored positions file entry, as the combination of zone_id, labels and last fetched timestamp.
The last timestamp fetched.
The set of fields being fetched.

Debug metrics

loki_source_cloudflare_target_entries_total (counter): Total number of successful entries sent via the cloudflare target.
loki_source_cloudflare_target_last_requested_end_timestamp (gauge): The last cloudflare request end timestamp fetched, for calculating how far behind the target is.

Example

This example pulls logs from Cloudflare’s API and forwards them to a loki.write component.

loki.source.cloudflare "dev" {
  zone_id   = env("CF_ZONE_ID")
  api_token = local.file.api.content

  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.cloudflare can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.docker

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.docker

loki.source.docker reads log entries from Docker containers and forwards them to other loki.* components. Each component can read from a single Docker daemon.

Multiple loki.source.docker components can be specified by giving them different labels.

Usage

loki.source.docker "LABEL" {
  host       = HOST
  targets    = TARGET_LIST
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.docker supports the following arguments:

Name	Type	Description	Default	Required
`host`	`string`	Address of the Docker daemon.		yes
`targets`	`list(map(string))`	List of containers to read logs from.		yes
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`labels`	`map(string)`	The default set of labels to apply on entries.	`"{}"`	no
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`"{}"`	no
`refresh_interval`	`duration`	The refresh interval to use when connecting to the Docker daemon over HTTP(S).	`"60s"`	no

Blocks

The following blocks are supported inside the definition of loki.source.docker:

Hierarchy	Block	Description	Required
client	client	HTTP client settings when connecting to the endpoint.	no
client > basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
client > authorization	authorization	Configure generic authorization to the endpoint.	no
client > oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
client > oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
client > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, client > basic_auth refers to an basic_auth block defined inside a client block.

These blocks are only applicable when connecting to a Docker daemon over HTTP or HTTPS and has no effect when connecting via a unix:/// socket

client block

The client block configures settings used to connect to HTTP(S) Docker daemons.

Name	Type	Description	Default	Required
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

bearer_token, bearer_token_file, basic_auth, authorization, and oauth2 are mutually exclusive, and only one can be provided inside of a http_client_config block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

basic_auth block

The basic_auth block configures basic authentication for HTTP(S) Docker daemons.

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

The authorization block configures custom authorization to use for the Docker daemon.

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

The oauth2 block configures OAuth2 authorization to use for the Docker daemon.

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

The tls_config block configures TLS settings for connecting to HTTPS Docker daemons.

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

loki.source.docker does not export any fields.

Component health

loki.source.docker is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.docker exposes some debug information per target:

Whether the target is ready to tail entries.
The labels associated with the target.
The most recent time a log line was read.

Debug metrics

loki_source_docker_target_entries_total (gauge): Total number of successful entries sent to the Docker target.
loki_source_docker_target_parsing_errors_total (gauge): Total number of parsing errors while receiving Docker messages.

Component behavior

The component uses its data path, a directory named after the domain’s fully qualified name, to store its positions file. The positions file is used to store read offsets, so that if a component or Grafana Agent restarts, loki.source.docker can pick up tailing from the same spot.

If the target’s argument contains multiple entries with the same container ID (for example as a result of discovery.docker picking up multiple exposed ports or networks), loki.source.docker will deduplicate them, and only keep the first of each container ID instances, based on the __meta_docker_container_id label. As such, the Docker daemon is queried for each container ID only once, and only one target will be available in the component’s debug info.

Example

This example collects log entries from the files specified in the targets argument and forwards them to a loki.write component to be written to Loki.

discovery.docker "linux" {
  host = "unix:///var/run/docker.sock"
}

loki.source.docker "default" {
  host       = "unix:///var/run/docker.sock"
  targets    = discovery.docker.linux.targets 
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.docker can accept arguments from the following components:

Components that export Targets
Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.file

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.file

loki.source.file reads log entries from files and forwards them to other loki.* components.

Multiple loki.source.file components can be specified by giving them different labels.

Note
loki.source.file does not handle file discovery. You can use local.file_match for file discovery. Refer to the File Globbing example for more information.

Usage

loki.source.file "LABEL" {
  targets    = TARGET_LIST
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.file supports the following arguments:

Name	Type	Description	Default	Required
`targets`	`list(map(string))`	List of files to read from.		yes
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`encoding`	`string`	The encoding to convert from when reading files.	`""`	no
`tail_from_end`	`bool`	Whether a log file should be tailed from the end if a stored position is not found.	`false`	no
`legacy_positions_file`	`string`	Allows conversion from legacy positions file.	`""`	no

The encoding argument must be a valid IANA encoding name. If not set, it defaults to UTF-8.

You can use the tail_from_end argument when you want to tail a large file without reading its entire content. When set to true, only new logs will be read, ignoring the existing ones.

Note
The legacy_positions_file argument is used when you are transitioning from legacy. The legacy positions file will be rewritten into the new format. This operation will only occur if the new positions file does not exist and the legacy_positions_file is valid. Once converted successfully, the legacy_positions_file will be deleted. If you add any labels before loki.source.file, then the positions file will conversion will not work. The legacy positions file did not have a concept of labels in the positions file, so the conversion assumes no labels.

Blocks

The following blocks are supported inside the definition of loki.source.file:

Hierarchy	Name	Description	Required
decompression	decompression	Configure reading logs from compressed files.	no
file_watch	file_watch	Configure how often files should be polled from disk for changes.	no

decompression block

The decompression block contains configuration for reading logs from compressed files. The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`bool`	Whether decompression is enabled.		yes
`initial_delay`	`duration`	Time to wait before starting to read from new compressed files.	0	no
`format`	`string`	Compression format.		yes

If you compress a file under a folder being scraped, loki.source.file might try to ingest your file before you finish compressing it. To avoid it, pick an initial_delay that is enough to avoid it.

Currently supported compression formats are:

gz - for gzip
z - for zlib
bz2 - for bzip2

The component can only support one compression format at a time, in order to handle multiple formats, you will need to create multiple components.

file_watch block

The file_watch block configures how often log files are polled from disk for changes. The following arguments are supported:

Name	Type	Description	Default	Required
`min_poll_frequency`	`duration`	Minimum frequency to poll for files.	250ms	no
`max_poll_frequency`	`duration`	Maximum frequency to poll for files.	250ms	no

If no file changes are detected, the poll frequency doubles until a file change is detected or the poll frequency reaches the max_poll_frequency.

If file changes are detected, the poll frequency is reset to min_poll_frequency.

Exported fields

loki.source.file does not export any fields.

Component health

loki.source.file is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.file exposes some target-level debug information per reader:

The tailed path.
Whether the reader is currently running.
What is the last recorded read offset in the positions file.

Debug metrics

loki_source_file_read_bytes_total (gauge): Number of bytes read.
loki_source_file_file_bytes_total (gauge): Number of bytes total.
loki_source_file_read_lines_total (counter): Number of lines read.
loki_source_file_encoding_failures_total (counter): Number of encoding failures.
loki_source_file_files_active_total (gauge): Number of active files.

Component behavior

If the decompression feature is deactivated, the component will continuously monitor and ’tail’ the files. In this mode, upon reaching the end of a file, the component remains active, awaiting and reading new entries in real-time as they are appended.

Each element in the list of targets as a set of key-value pairs called labels. The set of targets can either be static, or dynamically provided periodically by a service discovery component. The special label __path__ must always be present and must point to the absolute path of the file to read from.

The __path__ value is available as the filename label to each log entry the component reads. All other labels starting with a double underscore are considered internal and are removed from the log entries before they’re passed to other loki.* components.

The data path is inside the directory configured by the --storage.path command line argument.

If a file is removed from the targets list, its positions file entry is also removed. When it’s added back on, loki.source.file starts reading it from the beginning.

Examples

Static targets

This example collects log entries from the files specified in the targets argument and forwards them to a loki.write component to be written to Loki.

loki.source.file "tmpfiles" {
  targets    = [
    {__path__ = "/tmp/foo.txt", "color" = "pink"},
    {__path__ = "/tmp/bar.txt", "color" = "blue"},
    {__path__ = "/tmp/baz.txt", "color" = "grey"},
  ]
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

File globbing

This example collects log entries from the files matching *.log pattern using local.file_match component. When files appear or disappear, the list of targets will be updated accordingly.


local.file_match "logs" {
  path_targets = [
    {__path__ = "/tmp/*.log"},
  ]
}

loki.source.file "tmpfiles" {
  targets    = local.file_match.logs.targets
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Decompression

This example collects log entries from the compressed files matching *.gz pattern using local.file_match component and the decompression configuration on the loki.source.file component.


local.file_match "logs" {
  path_targets = [
    {__path__ = "/tmp/*.gz"},
  ]
}

loki.source.file "tmpfiles" {
  targets    = local.file_match.logs.targets
  forward_to = [loki.write.local.receiver]
  decompression {
    enabled       = true
    initial_delay = "10s"
    format        = "gz"
  }
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.file can accept arguments from the following components:

Components that export Targets
Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.gcplog

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.gcplog

loki.source.gcplog retrieves logs from cloud resources such as GCS buckets, load balancers, or Kubernetes clusters running on GCP by making use of Pub/Sub subscriptions.

The component uses either the ‘push’ or ‘pull’ strategy to retrieve log entries and forward them to the list of receivers in forward_to.

Multiple loki.source.gcplog components can be specified by giving them different labels.

Usage

loki.source.gcplog "LABEL" {
  pull {
    project_id   = "PROJECT_ID"
    subscription = "SUB_ID"
  }

  forward_to = RECEIVER_LIST
}

Arguments

loki.source.gcplog supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	“{}”	no

Blocks

The following blocks are supported inside the definition of loki.source.gcplog:

Hierarchy	Name	Description	Required
pull	pull	Configures a target to pull logs from a GCP Pub/Sub subscription.	no
push	push	Configures a server to receive logs as GCP Pub/Sub push requests.	no
push > http	http	Configures the HTTP server that receives requests when using the `push` mode.	no
push > grpc	grpc	Configures the gRPC server that receives requests when using the `push` mode.	no

The pull and push inner blocks are mutually exclusive; a component must contain exactly one of the two in its definition. The http and grpc block are just used when the push block is configured.

pull block

The pull block defines which GCP project ID and subscription to read log entries from.

The following arguments can be used to configure the pull block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`project_id`	`string`	The GCP project id the subscription belongs to.		yes
`subscription`	`string`	The subscription to pull logs from.		yes
`labels`	`map(string)`	Additional labels to associate with incoming logs.	`"{}"`	no
`use_incoming_timestamp`	`bool`	Whether to use the incoming log timestamp.	`false`	no
`use_full_line`	`bool`	Send the full line from Cloud Logging even if `textPayload` is available.	`false`	no

To make use of the pull strategy, the GCP project must have been configured to forward its cloud resource logs onto a Pub/Sub topic for loki.source.gcplog to consume.

Typically, the host system also needs to have its GCP credentials configured. One way to do it is to point the GOOGLE_APPLICATION_CREDENTIALS environment variable to the location of a credential configuration JSON file or a service account key.

push block

The push block defines the configuration of the server that receives push requests from GCP’s Pub/Sub servers.

The following arguments can be used to configure the push block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`graceful_shutdown_timeout`	`duration`	Timeout for servers graceful shutdown. If configured, should be greater than zero.	“30s”	no
`push_timeout`	`duration`	Sets a maximum processing time for each incoming GCP log entry.	`"0s"`	no
`labels`	`map(string)`	Additional labels to associate with incoming entries.	`"{}"`	no
`use_incoming_timestamp`	`bool`	Whether to use the incoming entry timestamp.	`false`	no
`use_full_line`	`bool`	Send the full line from Cloud Logging even if `textPayload` is available. By default, if `textPayload` is present in the line, then it’s used as log line	`false`	no

The server listens for POST requests from GCP’s Push subscriptions on HOST:PORT/gcp/api/v1/push.

By default, for both strategies the component assigns the log entry timestamp as the time it was processed, except if use_incoming_timestamp is set to true.

The labels map is applied to every entry that passes through the component.

http

The http block configures the HTTP server.

You can use the following arguments to configure the http block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. Defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections.	`8080`	no
`server_idle_timeout`	`duration`	Idle timeout for HTTP server.	`"120s"`	no
`server_read_timeout`	`duration`	Read timeout for HTTP server.	`"30s"`	no
`server_write_timeout`	`duration`	Write timeout for HTTP server.	`"30s"`	no

grpc

The grpc block configures the gRPC server.

You can use the following arguments to configure the grpc block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. It defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections. Defaults to a random free port.	`0`	no
`max_connection_age_grace`	`duration`	An additive period after `max_connection_age` after which the connection is forcibly closed.	`"infinity"`	no
`max_connection_age`	`duration`	The duration for the maximum time a connection may exist before it is closed.	`"infinity"`	no
`max_connection_idle`	`duration`	The duration after which an idle connection is closed.	`"infinity"`	no
`server_max_concurrent_streams`	`int`	Limit on the number of concurrent streams for gRPC calls (0 = unlimited).	`100`	no
`server_max_recv_msg_size`	`int`	Limit on the size of a gRPC message this server can receive (bytes).	`4MB`	no
`server_max_send_msg_size`	`int`	Limit on the size of a gRPC message this server can send (bytes).	`4MB`	no

Exported fields

loki.source.gcplog does not export any fields.

Component health

loki.source.gcplog is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.gcplog exposes some debug information per gcplog listener:

The configured strategy.
Their label set.
When using a push strategy, the listen address.

Debug metrics

When using the pull strategy, the component exposes the following debug metrics:

loki_source_gcplog_pull_entries_total (counter): Number of entries received by the gcplog target.
loki_source_gcplog_pull_parsing_errors_total (counter): Total number of parsing errors while receiving gcplog messages.
loki_source_gcplog_pull_last_success_scrape (gauge): Timestamp of target’s last successful poll.

When using the push strategy, the component exposes the following debug metrics:

loki_source_gcplog_push_entries_total (counter): Number of entries received by the gcplog target.
loki_source_gcplog_push_entries_total (counter): Number of parsing errors while receiving gcplog messages.

Example

This example listens for GCP Pub/Sub PushRequests on 0.0.0.0:8080 and forwards them to a loki.write component.

loki.source.gcplog "local" {
  push {}

  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

On the other hand, if we need the server to listen on 0.0.0.0:4040, and forwards them to a loki.write component.

loki.source.gcplog "local" {
  push {
    http {
        listen_port = 4040
    }
  }

  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.gcplog can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.gelf

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.gelf

loki.source.gelf reads Graylog Extended Long Format (GELF) logs from a UDP listener and forwards them to other loki.* components.

Multiple loki.source.gelf components can be specified by giving them different labels and ports.

Usage

loki.source.gelf "LABEL" {
  forward_to    = RECEIVER_LIST
}

Arguments

The component starts a new UDP listener and fans out log entries to the list of receivers passed in forward_to.

loki.source.gelf supports the following arguments:

Name	Type	Description	Default	Required
`listen_address`	`string`	UDP address and port to listen for Graylog messages.	`0.0.0.0:12201`	no
`use_incoming_timestamp`	`bool`	When false, assigns the current timestamp to the log when it was processed	`false`	no
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	“{}”	no

NOTE: GELF logs can be sent uncompressed or compressed with GZIP or ZLIB. A job label is added with the full name of the component loki.source.gelf.LABEL.

The relabel_rules argument can make use of the rules export from a loki.relabel component to apply one or more relabling rules to log entries before they’re forward to the list of receivers specified in forward_to.

Incoming messages have the following internal labels available:

__gelf_message_level: The GELF level as a string.
__gelf_message_host: The host sending the GELF message.
__gelf_message_host: The GELF level message version sent by the client.
__gelf_message_facility: The GELF facility.

All labels starting with __ are removed prior to forwarding log entries. To keep these labels, relabel them using a loki.relabel component and pass its rules export to the relabel_rules argument.

Component health

loki.source.gelf is only reported as unhealthy if given an invalid configuration.

Debug Metrics

gelf_target_entries_total (counter): Total number of successful entries sent to the GELF target.
gelf_target_parsing_errors_total (counter): Total number of parsing errors while receiving GELF messages.

Example

loki.relabel "gelf" {
  rule {
    source_labels = ["__gelf_message_host"]
    target_label  = "host"
  }
}

loki.source.gelf "listen"  {
  forward_to    = [loki.write.endpoint.receiver]
  relabel_rules = loki.relabel.gelf.rules
}

loki.write "endpoint" {
  endpoint {
    url ="loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.gelf can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.heroku

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.heroku

loki.source.heroku listens for Heroku messages over TCP connections and forwards them to other loki.* components.

The component starts a new heroku listener for the given listener block and fans out incoming entries to the list of receivers in forward_to.

Before using loki.source.heroku, Heroku should be configured with the URL where the Agent will be listening. Follow the steps in Heroku HTTPS Drain docs for using the Heroku CLI with a command like the following:

heroku drains:add [http|https]://HOSTNAME:PORT/heroku/api/v1/drain -a HEROKU_APP_NAME

Multiple loki.source.heroku components can be specified by giving them different labels.

Usage

loki.source.heroku "LABEL" {
    http {
        listen_address = "LISTEN_ADDRESS"
        listen_port    = LISTEN_PORT
    }
    forward_to = RECEIVER_LIST
}

Arguments

loki.source.heroku supports the following arguments:

Name	Type	Description	Default	Required
`use_incoming_timestamp`	`bool`	Whether or not to use the timestamp received from Heroku.	`false`	no
`labels`	`map(string)`	The labels to associate with each received Heroku record.	`{}`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no
`graceful_shutdown_timeout`	`duration`	Timeout for servers graceful shutdown. If configured, should be greater than zero.	“30s”	no

Blocks

The following blocks are supported inside the definition of loki.source.heroku:

Hierarchy	Name	Description	Required
`http`	http	Configures the HTTP server that receives requests.	no
`grpc`	grpc	Configures the gRPC server that receives requests.	no

http

The http block configures the HTTP server.

You can use the following arguments to configure the http block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. Defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections.	`8080`	no
`server_idle_timeout`	`duration`	Idle timeout for HTTP server.	`"120s"`	no
`server_read_timeout`	`duration`	Read timeout for HTTP server.	`"30s"`	no
`server_write_timeout`	`duration`	Write timeout for HTTP server.	`"30s"`	no

grpc

The grpc block configures the gRPC server.

You can use the following arguments to configure the grpc block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. It defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections. Defaults to a random free port.	`0`	no
`max_connection_age_grace`	`duration`	An additive period after `max_connection_age` after which the connection is forcibly closed.	`"infinity"`	no
`max_connection_age`	`duration`	The duration for the maximum time a connection may exist before it is closed.	`"infinity"`	no
`max_connection_idle`	`duration`	The duration after which an idle connection is closed.	`"infinity"`	no
`server_max_concurrent_streams`	`int`	Limit on the number of concurrent streams for gRPC calls (0 = unlimited).	`100`	no
`server_max_recv_msg_size`	`int`	Limit on the size of a gRPC message this server can receive (bytes).	`4MB`	no
`server_max_send_msg_size`	`int`	Limit on the size of a gRPC message this server can send (bytes).	`4MB`	no

Labels

The labels map is applied to every message that the component reads.

The following internal labels all prefixed with __ are available but will be discarded if not relabeled:

__heroku_drain_host
__heroku_drain_app
__heroku_drain_proc
__heroku_drain_log_id

All url query params will be translated to __heroku_drain_param_<name>

If the X-Scope-OrgID header is set it will be translated to __tenant_id__

Exported fields

loki.source.heroku does not export any fields.

Component health

loki.source.heroku is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.heroku exposes some debug information per Heroku listener:

Whether the listener is currently running.
The listen address.

Debug metrics

loki_source_heroku_drain_entries_total (counter): Number of successful entries received by the Heroku target.
loki_source_heroku_drain_parsing_errors_total (counter): Number of parsing errors while receiving Heroku messages.

Example

This example listens for Heroku messages over TCP in the specified port and forwards them to a loki.write component using the Heroku timestamp.

loki.source.heroku "local" {
    http {
        listen_address = "0.0.0.0"
        listen_port    = 4040
    }
    use_incoming_timestamp = true
    labels                 = {component = "loki.source.heroku"}
    forward_to             = [loki.write.local.receiver]
}

loki.write "local" {
    endpoint {
        url = "loki:3100/api/v1/push"
    }
}

When using the default http block settings, the server listen for new connection on port 8080.

loki.source.heroku "local" {
    use_incoming_timestamp = true
    labels                 = {component = "loki.source.heroku"}
    forward_to             = [loki.write.local.receiver]
}

loki.write "local" {
    endpoint {
        url = "loki:3100/api/v1/push"
    }
}

Compatible components

loki.source.heroku can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.journal

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.journal

loki.source.journal reads from the systemd journal and forwards them to other loki.* components.

Multiple loki.source.journal components can be specified by giving them different labels.

Usage

loki.source.journal "LABEL" {
  forward_to    = RECEIVER_LIST
}

Arguments

The component starts a new journal reader and fans out log entries to the list of receivers passed in forward_to.

loki.source.journal supports the following arguments:

Name	Type	Description	Default	Required
`format_as_json`	`bool`	Whether to forward the original journal entry as JSON.	`false`	no
`max_age`	`duration`	The oldest relative time from process start that will be read.	`"7h"`	no
`path`	`string`	Path to a directory to read entries from.	`""`	no
`matches`	`string`	Journal matches to filter. The `+` character is not supported, only logical AND matches will be added.	`""`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no
`labels`	`map(string)`	The labels to apply to every log coming out of the journal.	`{}`	no

NOTE: A job label is added with the full name of the component loki.source.journal.LABEL.

When the format_as_json argument is true, log messages are passed through as JSON with all of the original fields from the journal entry. Otherwise, the log message is taken from the content of the MESSAGE field from the journal entry.

When the path argument is empty, /var/log/journal and /run/log/journal will be used for discovering journal entries.

The relabel_rules argument can make use of the rules export value from a loki.relabel component to apply one or more relabeling rules to log entries before they’re forwarded to the list of receivers in forward_to.

All messages read from the journal include internal labels following the pattern of __journal_FIELDNAME and will be dropped before sending to the list of receivers specified in forward_to. To keep these labels, use the relabel_rules argument and relabel them to not be prefixed with __.

NOTE: many field names from journald start with an _, such as _systemd_unit. The final internal label name would be __journal__systemd_unit, with two underscores between __journal and systemd_unit.

Component health

loki.source.journal is only reported as unhealthy if given an invalid configuration.

Debug Metrics

agent_loki_source_journal_target_parsing_errors_total (counter): Total number of parsing errors while reading journal messages.
agent_loki_source_journal_target_lines_total (counter): Total number of successful journal lines read.

Example

loki.relabel "journal" {
  forward_to = []

  rule {
    source_labels = ["__journal__systemd_unit"]
    target_label  = "unit"
  }
}

loki.source.journal "read"  {
  forward_to    = [loki.write.endpoint.receiver]
  relabel_rules = loki.relabel.journal.rules
  labels        = {component = "loki.source.journal"}
}

loki.write "endpoint" {
  endpoint {
    url ="loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.journal can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.kafka

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.kafka

loki.source.kafka reads messages from Kafka using a consumer group and forwards them to other loki.* components.

The component starts a new Kafka consumer group for the given arguments and fans out incoming entries to the list of receivers in forward_to.

Before using loki.source.kafka, Kafka should have at least one producer writing events to at least one topic. Follow the steps in the Kafka Quick Start to get started with Kafka.

Multiple loki.source.kafka components can be specified by giving them different labels.

Usage

loki.source.kafka "LABEL" {
  brokers    = BROKER_LIST
  topics     = TOPIC_LIST
  forward_to = RECEIVER_LIST
}

Arguments

loki.source.kafka supports the following arguments:

Name	Type	Description	Default	Required
`brokers`	`list(string)`	The list of brokers to connect to Kafka.		yes
`topics`	`list(string)`	The list of Kafka topics to consume.		yes
`group_id`	`string`	The Kafka consumer group id.	`"loki.source.kafka"`	no
`assignor`	`string`	The consumer group rebalancing strategy to use.	`"range"`	no
`version`	`string`	Kafka version to connect to.	`"2.2.1"`	no
`use_incoming_timestamp`	`bool`	Whether or not to use the timestamp received from Kafka.	`false`	no
`labels`	`map(string)`	The labels to associate with each received Kafka event.	`{}`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	`{}`	no

assignor values can be either "range", "roundrobin", or "sticky".

Labels from the labels argument are applied to every message that the component reads.

The relabel_rules field can make use of the rules export value from a loki.relabel component to apply one or more relabeling rules to log entries before they’re forwarded to the list of receivers in forward_to.

In addition to custom labels, the following internal labels prefixed with __ are available:

__meta_kafka_message_key
__meta_kafka_message_offset
__meta_kafka_topic
__meta_kafka_partition
__meta_kafka_member_id
__meta_kafka_group_id

Blocks

The following blocks are supported inside the definition of loki.source.kafka:

Hierarchy	Name	Description	Required
authentication	authentication	Optional authentication configuration with Kafka brokers.	no
authentication > tls_config	tls_config	Optional authentication configuration with Kafka brokers.	no
authentication > sasl_config	sasl_config	Optional authentication configuration with Kafka brokers.	no
authentication > sasl_config > tls_config	tls_config	Optional authentication configuration with Kafka brokers.	no
authentication > sasl_config > oauth_config	oauth_config	Optional authentication configuration with Kafka brokers.	no

authentication block

The authentication block defines the authentication method when communicating with the Kafka event brokers.

Name	Type	Description	Default	Required
`type`	`string`	Type of authentication.	`"none"`	no

type supports the values "none", "ssl", and "sasl". If "ssl" is used, you must set the tls_config block. If "sasl" is used, you must set the sasl_config block.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

sasl_config block

The sasl_config block defines the listen address and port where the listener expects Kafka messages to be sent to.

Name	Type	Description	Default	Required
`mechanism`	`string`	Specifies the SASL mechanism the client uses to authenticate with the broker.	`"PLAIN""`	no
`user`	`string`	The user name to use for SASL authentication.	`""`	no
`password`	`secret`	The password to use for SASL authentication.	`""`	no
`use_tls`	`bool`	If true, SASL authentication is executed over TLS.	`false`	no

oauth_config block

The oauth_config is required when the SASL mechanism is set to OAUTHBEARER.

Name	Type	Description	Default	Required
`token_provider`	`string`	The OAuth provider to be used. The only supported provider is `azure`.	`""`	yes
`scopes`	`list(string)`	The scopes to set in the access token	`[]`	yes

Exported fields

loki.source.kafka does not export any fields.

Component health

loki.source.kafka is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.kafka does not expose additional debug info.

Example

This example consumes Kafka events from the specified brokers and topics then forwards them to a loki.write component using the Kafka timestamp.

loki.source.kafka "local" {
  brokers                = ["localhost:9092"]
  topics                 = ["quickstart-events"]
  labels                 = {component = "loki.source.kafka"}
  forward_to             = [loki.relabel.kafka.receiver]
  use_incoming_timestamp = true
  relabel_rules          = loki.relabel.kafka.rules
}

loki.relabel "kafka" {
  forward_to      = [loki.write.local.receiver]

  rule {
    source_labels = ["__meta_kafka_topic"]
    target_label  = "topic"
  }
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.kafka can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.kubernetes

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.kubernetes

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

loki.source.kubernetes tails logs from Kubernetes containers using the Kubernetes API. It has the following benefits over loki.source.file:

It works without a privileged container.
It works without a root user.
It works without needing access to the filesystem of the Kubernetes node.
It doesn’t require a DaemonSet to collect logs, so one Grafana Agent could collect logs for the whole cluster.

NOTE: Because loki.source.kubernetes uses the Kubernetes API to tail logs, it uses more network traffic and CPU consumption of Kubelets than loki.source.file.

Multiple loki.source.kubernetes components can be specified by giving them different labels.

Usage

loki.source.kubernetes "LABEL" {
  targets    = TARGET_LIST
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.kubernetes supports the following arguments:

Name	Type	Description	Default	Required
`targets`	`list(map(string))`	List of files to read from.		yes
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes

Each target in targets must have the following labels:

__meta_kubernetes_namespace or __pod_namespace__ to specify the namespace of the pod to tail.
__meta_kubernetes_pod_name or __pod_name__ to specify the name of the pod to tail.
__meta_kubernetes_pod_container_name or __pod_container_name__ to specify the container within the pod to tail.
__meta_kubernetes_pod_uid or __pod_uid__ to specify the UID of the pod to tail.

By default, all of these labels are present when the output discovery.kubernetes is used.

A log tailer is started for each unique target in targets. Log tailers will reconnect with exponential backoff to Kubernetes if the log stream returns before the container has permanently terminated.

Blocks

The following blocks are supported inside the definition of loki.source.kubernetes:

Hierarchy	Block	Description	Required
client	client	Configures Kubernetes client used to tail logs.	no
client > basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
client > authorization	authorization	Configure generic authorization to the endpoint.	no
client > oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
client > oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
client > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
clustering	clustering	Configure the component for when Grafana Agent Flow is running in clustered mode.	no

The > symbol indicates deeper levels of nesting. For example, client > basic_auth refers to a basic_auth block defined inside a client block.

client block

The client block configures the Kubernetes client used to tail logs from containers. If the client block isn’t provided, the default in-cluster configuration with the service account of the running Grafana Agent pod is used.

The following arguments are supported:

Name	Type	Description	Default	Required
`api_server`	`string`	URL of the Kubernetes API server.		no
`kubeconfig_file`	`string`	Path of the `kubeconfig` file to use for connecting to Kubernetes.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

clustering block

Name	Type	Description	Default	Required
`enabled`	`bool`	Distribute log collection with other cluster nodes.		yes

When Grafana Agent is using clustering, and enabled is set to true, then this loki.source.kubernetes component instance opts-in to participating in the cluster to distribute the load of log collection between all cluster nodes.

If Grafana Agent is not running in clustered mode, then the block is a no-op and loki.source.kubernetes collects logs from every target it receives in its arguments.

Exported fields

loki.source.kubernetes does not export any fields.

Component health

loki.source.kubernetes is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.kubernetes exposes some target-level debug information per target:

The labels associated with the target.
The full set of labels which were found during service discovery.
The most recent time a log line was read and forwarded to the next components in the pipeline.
The most recent error from tailing, if any.

Debug metrics

loki.source.kubernetes does not expose any component-specific debug metrics.

Example

This example collects logs from all Kubernetes pods and forwards them to a loki.write component so they are written to Loki.

discovery.kubernetes "pods" {
  role = "pod"
}

loki.source.kubernetes "pods" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = env("LOKI_URL")
  }
}

Compatible components

loki.source.kubernetes can accept arguments from the following components:

Components that export Targets
Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.kubernetes_events

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.kubernetes_events

loki.source.kubernetes_events tails events from the Kubernetes API and converts them into log lines to forward to other loki components.

Multiple loki.source.kubernetes_events components can be specified by giving them different labels.

Usage

loki.source.kubernetes_events "LABEL" {
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.kubernetes_events supports the following arguments:

Name	Type	Description	Default	Required
`job_name`	`string`	Value to use for `job` label for generated logs.	`"loki.source.kubernetes_events"`	no
`log_format`	`string`	Format of the log.	`"logfmt"`	no
`namespaces`	`list(string)`	Namespaces to watch for Events in.	`[]`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes

By default, loki.source.kubernetes_events will watch for events in all namespaces. A list of explicit namespaces to watch can be provided in the namespaces argument.

By default, the generated log lines will be in the logfmt format. Use the log_format argument to change it to json. These formats are also names of LogQL parsers, which can be used for processing the logs.

NOTE: When watching all namespaces, Grafana Agent Flow must have permissions to watch events at the cluster scope (such as using a ClusterRoleBinding). If an explicit list of namespaces is provided, Grafana Agent Flow only needs permissions to watch events for those namespaces.

Log lines generated by loki.source.kubernetes_events have the following labels:

namespace: Namespace of the Kubernetes object involved in the event.
job: Value specified by the job_name argument.
instance: Value matching the component ID.

If job_name argument is the empty string, the component will fail to load. To remove the job label, forward the output of loki.source.kubernetes_events to a loki.relabel component.

For compatibility with the eventhandler integration from static mode, job_name can be set to "integrations/kubernetes/eventhandler".

Blocks

The following blocks are supported inside the definition of loki.source.kubernetes_events:

Hierarchy	Block	Description	Required
client	client	Configures Kubernetes client used to tail logs.	no
client > basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
client > authorization	authorization	Configure generic authorization to the endpoint.	no
client > oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
client > oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
client > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, client > basic_auth refers to a basic_auth block defined inside a client block.

client block

The following arguments are supported:

Name	Type	Description	Default	Required
`api_server`	`string`	URL of the Kubernetes API server.		no
`kubeconfig_file`	`string`	Path of the `kubeconfig` file to use for connecting to Kubernetes.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

loki.source.kubernetes_events does not export any fields.

Component health

loki.source.kubernetes_events is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.kubernetes_events exposes the most recently read timestamp for events in each watched namespace.

Debug metrics

loki.source.kubernetes_events does not expose any component-specific debug metrics.

Component behavior

The data path is inside the directory configured by the --storage.path command line argument.

In the Static mode’s eventhandler integration, a cache_path argument is used to configure a positions file. In Flow mode, this argument is no longer necessary.

Example

This example collects watches events in the kube-system namespace and forwards them to a loki.write component so they are written to Loki.

loki.source.kubernetes_events "example" {
  // Only watch for events in the kube-system namespace.
  namespaces = ["kube-system"]

  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = env("LOKI_URL")
  }
}

Compatible components

loki.source.kubernetes_events can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.podlogs

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.podlogs

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

loki.source.podlogs discovers PodLogs resources on Kubernetes and, using the Kubernetes API, tails logs from Kubernetes containers of Pods specified by the discovered them.

loki.source.podlogs is similar to loki.source.kubernetes, but uses custom resources rather than being fed targets from another Flow component.

NOTE: Unlike loki.source.kubernetes, it is not possible to distribute responsibility of collecting logs across multiple Grafana Agents. To avoid collecting duplicate logs, only one Grafana Agent should be running a loki.source.podlogs component.

NOTE: Because loki.source.podlogs uses the Kubernetes API to tail logs, it uses more network traffic and CPU consumption of Kubelets than loki.source.file.

Multiple loki.source.podlogs components can be specified by giving them different labels.

Usage

loki.source.podlogs "LABEL" {
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.podlogs supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes

loki.source.podlogs searches for PodLogs resources on Kubernetes. Each PodLogs resource describes a set of pods to tail logs from.

PodLogs custom resource

The PodLogs resource describes a set of Pods to collect logs from.

NOTE: loki.source.podlogs looks for PodLogs of monitoring.grafana.com/v1alpha2, and is not compatible with PodLogs from the Grafana Agent Operator, which are version v1alpha1.

Field	Type	Description
`apiVersion`	string	`monitoring.grafana.com/v1alpha2`
`kind`	string	`PodLogs`
`metadata`	ObjectMeta	Metadata for the PodLogs.
`spec`	PodLogsSpec	Definition of what Pods to collect logs from.

PodLogsSpec

PodLogsSpec describes a set of Pods to collect logs from.

Field	Type	Description
`selector`	LabelSelector	Label selector of Pods to collect logs from.
`namespaceSelector`	LabelSelector	Label selector of Namespaces that Pods can be discovered in.
`relabelings`	RelabelConfig	Relabel rules to apply to discovered Pods.

If selector is left as the default value, all Pods are discovered. If namespaceSelector is left as the default value, all Namespaces are used for Pod discovery.

The relabelings field can be used to modify labels from discovered Pods. The following meta labels are available for relabeling:

__meta_kubernetes_namespace: The namespace of the Pod.
__meta_kubernetes_pod_name: The name of the Pod.
__meta_kubernetes_pod_ip: The pod IP of the Pod.
__meta_kubernetes_pod_label_<labelname>: Each label from the Pod.
__meta_kubernetes_pod_labelpresent_<labelname>: true for each label from the Pod.
__meta_kubernetes_pod_annotation_<annotationname>: Each annotation from the Pod.
__meta_kubernetes_pod_annotationpresent_<annotationname>: true for each annotation from the Pod.
__meta_kubernetes_pod_container_init: true if the container is an InitContainer.
__meta_kubernetes_pod_container_name: Name of the container.
__meta_kubernetes_pod_container_image: The image the container is using.
__meta_kubernetes_pod_ready: Set to true or false for the Pod’s ready state.
__meta_kubernetes_pod_phase: Set to Pending, Running, Succeeded, Failed or Unknown in the lifecycle.
__meta_kubernetes_pod_node_name: The name of the node the pod is scheduled onto.
__meta_kubernetes_pod_host_ip: The current host IP of the pod object.
__meta_kubernetes_pod_uid: The UID of the Pod.
__meta_kubernetes_pod_controller_kind: Object kind of the Pod’s controller.
__meta_kubernetes_pod_controller_name: Name of the Pod’s controller.

In addition to the meta labels, the following labels are exposed to tell loki.source.podlogs which container to tail:

__pod_namespace__: The namespace of the Pod.
__pod_name__: The name of the Pod.
__pod_container_name__: The container name within the Pod.
__pod_uid__: The UID of the Pod.

Blocks

The following blocks are supported inside the definition of loki.source.podlogs:

Hierarchy	Block	Description	Required
client	client	Configures Kubernetes client used to tail logs.	no
client > basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
client > authorization	authorization	Configure generic authorization to the endpoint.	no
client > oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
client > oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
client > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
selector	selector	Label selector for which `PodLogs` to discover.	no
selector > match_expression	match_expression	Label selector expression for which `PodLogs` to discover.	no
namespace_selector	selector	Label selector for which namespaces to discover `PodLogs` in.	no
namespace_selector > match_expression	match_expression	Label selector expression for which namespaces to discover `PodLogs` in.	no
clustering	clustering	Configure the component for when Grafana Agent is running in clustered mode.	no

The > symbol indicates deeper levels of nesting. For example, client > basic_auth refers to a basic_auth block defined inside a client block.

client block

The following arguments are supported:

Name	Type	Description	Default	Required
`api_server`	`string`	URL of the Kubernetes API server.		no
`kubeconfig_file`	`string`	Path of the `kubeconfig` file to use for connecting to Kubernetes.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

selector block

The selector block describes a Kubernetes label selector for PodLogs or Namespace discovery.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_labels`	`map(string)`	Label keys and values used to discover resources.	`{}`	no

When the match_labels argument is empty, all resources will be matched.

match_expression block

The match_expression block describes a Kubernetes label match expression for PodLogs or Namespace discovery.

The following arguments are supported:

Name	Type	Description	Required
`key`	`string`	The label name to match against.	yes
`operator`	`string`	The operator to use when matching.	yes
`values`	`list(string)`	The values used when matching.	no

The operator argument must be one of the following strings:

"In"
"NotIn"
"Exists"
"DoesNotExist"

Both selector and namespace_selector can make use of multiple match_expression inner blocks which are treated as AND clauses.

clustering block

Name	Type	Description	Default	Required
`enabled`	`bool`	Distribute log collection with other cluster nodes.		yes

When Grafana Agent Flow is using clustering, and enabled is set to true, then this loki.source.podlogs component instance opts-in to participating in the cluster to distribute the load of log collection between all cluster nodes.

If Grafana Agent Flow is not running in clustered mode, then the block is a no-op and loki.source.podlogs collects logs based on every PodLogs resource discovered.

Exported fields

loki.source.podlogs does not export any fields.

Component health

loki.source.podlogs is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.podlogs exposes some target-level debug information per target:

The labels associated with the target.
The full set of labels which were found during service discovery.
The most recent time a log line was read and forwarded to the next components in the pipeline.
The most recent error from tailing, if any.

Debug metrics

loki.source.podlogs does not expose any component-specific debug metrics.

Example

This example discovers all PodLogs resources and forwards collected logs to a loki.write component so they are written to Loki.

loki.source.podlogs "default" {
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = env("LOKI_URL")
  }
}

Compatible components

loki.source.podlogs can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.syslog

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.syslog

loki.source.syslog listens for syslog messages over TCP or UDP connections and forwards them to other loki.* components. The messages must be compliant with the RFC5424 format.

The component starts a new syslog listener for each of the given config blocks and fans out incoming entries to the list of receivers in forward_to.

Multiple loki.source.syslog components can be specified by giving them different labels.

Usage

loki.source.syslog "LABEL" {
  listener {
    address = "LISTEN_ADDRESS"
  }
  ...

  forward_to = RECEIVER_LIST
}

Arguments

loki.source.syslog supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`relabel_rules`	`RelabelRules`	Relabeling rules to apply on log entries.	“{}”	no

Blocks

The following blocks are supported inside the definition of loki.source.syslog:

Hierarchy	Name	Description	Required
listener	listener	Configures a listener for IETF Syslog (RFC5424) messages.	no
listener > tls_config	tls_config	Configures TLS settings for connecting to the endpoint for TCP connections.	no

The > symbol indicates deeper levels of nesting. For example, config > tls_config refers to a tls_config block defined inside a config block.

listener block

The listener block defines the listen address and protocol where the listener expects syslog messages to be sent to, as well as its behavior when receiving messages.

The following arguments can be used to configure a listener. Only the address field is required and any omitted fields take their default values.

Name	Type	Description	Default	Required
`address`	`string`	The `<host:port>` address to listen to for syslog messages.		yes
`protocol`	`string`	The protocol to listen to for syslog messages. Must be either `tcp` or `udp`.	`tcp`	no
`idle_timeout`	`duration`	The idle timeout for tcp connections.	`"120s"`	no
`label_structured_data`	`bool`	Whether to translate syslog structured data to loki labels.	`false`	no
`labels`	`map(string)`	The labels to associate with each received syslog record.	`{}`	no
`use_incoming_timestamp`	`bool`	Whether to set the timestamp to the incoming syslog record timestamp.	`false`	no
`use_rfc5424_message`	`bool`	Whether to forward the full RFC5424-formatted syslog message.	`false`	no
`max_message_length`	`int`	The maximum limit to the length of syslog messages.	`8192`	no

By default, the component assigns the log entry timestamp as the time it was processed.

The labels map is applied to every message that the component reads.

All header fields from the parsed RFC5424 messages are brought in as internal labels, prefixed with __syslog_.

If label_structured_data is set, structured data in the syslog header is also translated to internal labels in the form of __syslog_message_sd_<ID>_<KEY>. For example, a structured data entry of [example@99999 test="yes"] becomes the label __syslog_message_sd_example_99999_test with the value "yes".

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

loki.source.syslog does not export any fields.

Component health

loki.source.syslog is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.syslog exposes some debug information per syslog listener:

Whether the listener is currently running.
The listen address.
The labels that the listener applies to incoming log entries.

Debug metrics

loki_source_syslog_entries_total (counter): Total number of successful entries sent to the syslog component.
loki_source_syslog_parsing_errors_total (counter): Total number of parsing errors while receiving syslog messages.
loki_source_syslog_empty_messages_total (counter): Total number of empty messages received from the syslog component.

Example

This example listens for Syslog messages in valid RFC5424 format over TCP and UDP in the specified ports and forwards them to a loki.write component.

loki.source.syslog "local" {
  listener {
    address  = "127.0.0.1:51893"
    labels   = { component = "loki.source.syslog", protocol = "tcp" }
  }

  listener {
    address  = "127.0.0.1:51898"
    protocol = "udp"
    labels   = { component = "loki.source.syslog", protocol = "udp"}
  }

  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}

Compatible components

loki.source.syslog can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.source.windowsevent

Fri, 10 Apr 2026 07:58:08 +0000

loki.source.windowsevent

loki.source.windowsevent reads events from Windows Event Logs and forwards them to other loki.* components.

Multiple loki.source.windowsevent components can be specified by giving them different labels.

Usage

loki.source.windowsevent "LABEL" {
  eventlog_name = EVENTLOG_NAME
  forward_to    = RECEIVER_LIST
}

Arguments

The component starts a new reader and fans out log entries to the list of receivers passed in forward_to.

loki.source.windowsevent supports the following arguments:

Name	Type	Description	Default	Required
`locale`	`number`	Locale ID for event rendering. 0 default is Windows Locale.	`0`	no
`eventlog_name`	`string`	Event log to read from.		See below.
`xpath_query`	`string`	Event log to read from.	`"*"`	See below.
`bookmark_path`	`string`	Keeps position in event log.	`"DATA_PATH/bookmark.xml"`	no
`poll_interval`	`duration`	How often to poll the event log.	`"3s"`	no
`exclude_event_data`	`bool`	Exclude event data.	`false`	no
`exclude_user_data`	`bool`	Exclude user data.	`false`	no
`exclude_event_message`	`bool`	Exclude the human-friendly event message.	`false`	no
`use_incoming_timestamp`	`bool`	When false, assigns the current timestamp to the log when it was processed.	`false`	no
`forward_to`	`list(LogsReceiver)`	List of receivers to send log entries to.		yes
`labels`	`map(string)`	The labels to associate with incoming logs.		no
`legacy_bookmark_path`	`string`	The location of the Grafana Agent Static bookmark path.	``	no

NOTE: eventlog_name is required if xpath_query does not specify the event log. You can define xpath_query in short or xml form. When using the XML form you can specify event_log in the xpath_query. If using short form, you must define eventlog_name.

Note
legacy_bookmark_path is used to convert the Grafana Agent Static to a Grafana Agent Flow bookmark, if bookmark_path does not exist.

Component health

loki.source.windowsevent is only reported as unhealthy if given an invalid configuration.

Example

This example collects log entries from the Event Log specified in eventlog_name and forwards them to a loki.write component so they are written to Loki.

loki.source.windowsevent "application"  {
    eventlog_name = "Application"
    forward_to = [loki.write.endpoint.receiver]
}

loki.write "endpoint" {
    endpoint {
        url ="loki:3100/api/v1/push"
    }
}

Compatible components

loki.source.windowsevent can accept arguments from the following components:

Components that export Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

loki.write

Fri, 10 Apr 2026 07:58:08 +0000

loki.write

loki.write receives log entries from other loki components and sends them over the network using Loki’s logproto format.

Multiple loki.write components can be specified by giving them different labels.

Usage

loki.write "LABEL" {
  endpoint {
    url = REMOTE_WRITE_URL
  }
}

Arguments

loki.write supports the following arguments:

Name	Type	Description	Default	Required
`max_streams`	`int`	Maximum number of active streams.	0 (no limit)	no
`external_labels`	`map(string)`	Labels to add to logs sent over the network.		no

Blocks

The following blocks are supported inside the definition of loki.write:

Hierarchy	Block	Description	Required
endpoint	endpoint	Location to send logs to.	no
wal	wal	Write-ahead log configuration.	no
endpoint > basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
endpoint > authorization	authorization	Configure generic authorization to the endpoint.	no
endpoint > oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
endpoint > oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
endpoint > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
endpoint > queue_config	queue_config	When WAL is enabled, configures the queue client.	no

The > symbol indicates deeper levels of nesting. For example, endpoint > basic_auth refers to a basic_auth block defined inside an endpoint block.

endpoint block

The endpoint block describes a single location to send logs to. Multiple endpoint blocks can be provided to send logs to multiple locations.

The following arguments are supported:

Name	Type	Description	Default	Required
`url`	`string`	Full URL to send logs to.		yes
`name`	`string`	Optional name to identify this endpoint with.		no
`headers`	`map(string)`	Extra headers to deliver with the request.		no
`batch_wait`	`duration`	Maximum amount of time to wait before sending a batch.	`"1s"`	no
`batch_size`	`string`	Maximum batch size of logs to accumulate before sending.	`"1MiB"`	no
`remote_timeout`	`duration`	Timeout for requests made to the URL.	`"10s"`	no
`tenant_id`	`string`	The tenant ID used by default to push logs.		no
`min_backoff_period`	`duration`	Initial backoff time between retries.	`"500ms"`	no
`max_backoff_period`	`duration`	Maximum backoff time between retries.	`"5m"`	no
`max_backoff_retries`	`int`	Maximum number of retries.	10	no
`retry_on_http_429`	`bool`	Retry when an HTTP 429 status code is received.	`true`	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

If no tenant_id is provided, the component assumes that the Loki instance at endpoint is running in single-tenant mode and no X-Scope-OrgID header is sent.

When multiple endpoint blocks are provided, the loki.write component creates a client for each. Received log entries are fanned-out to these clients in succession. That means that if one client is bottlenecked, it may impact the rest.

Endpoints can be named for easier identification in debug metrics by using the name argument. If the name argument isn’t provided, a name is generated based on a hash of the endpoint settings.

The retry_on_http_429 argument specifies whether HTTP 429 status code responses should be treated as recoverable errors; other HTTP 4xx status code responses are never considered recoverable errors. When retry_on_http_429 is enabled, the retry mechanism will be governed by the backoff configuration specified through min_backoff_period, max_backoff_period and max_backoff_retries attributes.

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

queue_config block (experimental)

The optional queue_config block configures, when WAL is enabled (see Write-Ahead block), how the underlying client queues batches of logs to be sent to Loki.

The following arguments are supported:

Name	Type	Description	Default	Required
`capacity`	`string`	Controls the size of the underlying send queue buffer. This setting should be considered a worst-case scenario of memory consumption, in which all enqueued batches are full.	`10MiB`	no
`drain_timeout`	`duration`	Configures the maximum time the client can take to drain the send queue upon shutdown. During that time, it will enqueue pending batches and drain the send queue sending each.	`"1m"`	no

wal block (experimental)

The optional wal block configures the Write-Ahead Log (WAL) used in the Loki remote-write client. To enable the WAL, you must include the wal block in your configuration. When the WAL is enabled, the log entries sent to the loki.write component are first written to a WAL under the dir directory and then read into the remote-write client. This process provides durability guarantees when an entry reaches this component. The client knows when to read from the WAL using the following two mechanisms:

The WAL-writer side of the loki.write component notifies the reader side that new data is available.
The WAL-reader side periodically checks if there is new data, increasing the wait time exponentially between min_read_frequency and max_read_frequency.

The WAL is located inside a component-specific directory relative to the storage path Grafana Agent Flow is configured to use. See the agent run documentation for how to change the storage path.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`bool`	Whether to enable the WAL.	false	no
`max_segment_age`	`duration`	Maximum time a WAL segment should be allowed to live. Segments older than this setting will be eventually deleted.	`"1h"`	no
`min_read_frequency`	`duration`	Minimum backoff time in the backup read mechanism.	`"250ms"`	no
`max_read_frequency`	`duration`	Maximum backoff time in the backup read mechanism.	`"1s"`	no
`drain_timeout`	`duration`	Maximum time the WAL drain procedure can take, before being forcefully stopped.	`"30s"`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`LogsReceiver`	A value that other components can use to send log entries to.

Component health

loki.write is only reported as unhealthy if given an invalid configuration.

Debug information

loki.write does not expose any component-specific debug information.

Debug metrics

loki_write_encoded_bytes_total (counter): Number of bytes encoded and ready to send.
loki_write_sent_bytes_total (counter): Number of bytes sent.
loki_write_dropped_bytes_total (counter): Number of bytes dropped because failed to be sent to the ingester after all retries.
loki_write_sent_entries_total (counter): Number of log entries sent to the ingester.
loki_write_dropped_entries_total (counter): Number of log entries dropped because they failed to be sent to the ingester after all retries.
loki_write_request_duration_seconds (histogram): Duration of sent requests.
loki_write_batch_retries_total (counter): Number of times batches have had to be retried.
loki_write_stream_lag_seconds (gauge): Difference between current time and last batch timestamp for successful sends.

Examples

The following examples show you how to create loki.write components that send log entries to different destinations.

Send log entries to a local Loki instance

You can create a loki.write component that sends your log entries to a local Loki instance:

loki.write "local" {
    endpoint {
        url = "http://loki:3100/loki/api/v1/push"
    }
}

Send log entries to a managed service

You can create a loki.write component that sends your log entries to a managed service, for example, Grafana Cloud. The Loki username and Grafana Cloud API Key are injected in this example through environment variables.

loki.write "default" {
    endpoint {
        url = "https://logs-xxx.grafana.net/loki/api/v1/push"
        basic_auth {
            username = env("LOKI_USERNAME")
            password = env("GRAFANA_CLOUD_API_KEY")
        }
    }
}

Technical details

loki.write uses snappy for compression.

Any labels that start with __ will be removed before sending to the endpoint.

Compatible components

loki.write has exports that can be consumed by the following components:

Components that consume Loki LogsReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

mimir.rules.kubernetes

Fri, 10 Apr 2026 07:58:08 +0000

mimir.rules.kubernetes

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

mimir.rules.kubernetes discovers PrometheusRule Kubernetes resources and loads them into a Mimir instance.

Multiple mimir.rules.kubernetes components can be specified by giving them different labels.
Kubernetes label selectors can be used to limit the Namespace and PrometheusRule resources considered during reconciliation.
Compatible with the Ruler APIs of Grafana Mimir, Grafana Cloud, and Grafana Enterprise Metrics.
Compatible with the PrometheusRule CRD from the prometheus-operator.
This component accesses the Kubernetes REST API from within a Pod.

NOTE: This component requires Role-based access control (RBAC) to be setup in Kubernetes in order for the Agent to access it via the Kubernetes REST API. For an example RBAC configuration please click here.

Usage

mimir.rules.kubernetes "LABEL" {
  address = MIMIR_RULER_URL
}

Arguments

mimir.rules.kubernetes supports the following arguments:

Name	Type	Description	Default	Required
`address`	`string`	URL of the Mimir ruler.		yes
`tenant_id`	`string`	Mimir tenant ID.		no
`use_legacy_routes`	`bool`	Whether to use [deprecated][gem-2_2] ruler API endpoints.	false	no
`prometheus_http_prefix`	`string`	Path prefix for [Mimir’s Prometheus endpoint][gem-path-prefix].	`/prometheus`	no
`sync_interval`	`duration`	Amount of time between reconciliations with Mimir.	“5m”	no
`mimir_namespace_prefix`	`string`	Prefix used to differentiate multiple Grafana Agent Flow deployments.	“agent”	no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether HTTP2 is supported for requests.	`true`	no
`follow_redirects`	`bool`	Whether redirects returned by the server should be followed.	`true`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no

At most, one of the following can be provided:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

If no tenant_id is provided, the component assumes that the Mimir instance at address is running in single-tenant mode and no X-Scope-OrgID header is sent.

The sync_interval argument determines how often Mimir’s ruler API is accessed to reload the current state of rules. Interaction with the Kubernetes API works differently. Updates are processed as events from the Kubernetes API server according to the informer pattern.

The mimir_namespace_prefix argument can be used to separate the rules managed by multiple Grafana Agent Flow deployments across your infrastructure. It should be set to a unique value for each deployment.

If use_legacy_routes is set to true, mimir.rules.kubernetes contacts Mimir on a /api/v1/rules endpoint.

If prometheus_http_prefix is set to /mimir, mimir.rules.kubernetes contacts Mimir on a /mimir/config/v1/rules endpoint. This is useful if you configure Mimir to use a different [prefix][gem-path-prefix] for its Prometheus endpoints than the default one.

prometheus_http_prefix is ignored if use_legacy_routes is set to true.

Blocks

The following blocks are supported inside the definition of mimir.rules.kubernetes:

Hierarchy	Block	Description	Required
rule_namespace_selector	label_selector	Label selector for `Namespace` resources.	no
rule_namespace_selector > match_expression	match_expression	Label match expression for `Namespace` resources.	no
rule_selector	label_selector	Label selector for `PrometheusRule` resources.	no
rule_selector > match_expression	match_expression	Label match expression for `PrometheusRule` resources.	no
basic_auth	basic_auth	Configure basic_auth for authenticating to the endpoint.	no
authorization	authorization	Configure generic authorization to the endpoint.	no
oauth2	oauth2	Configure OAuth2 for authenticating to the endpoint.	no
oauth2 > tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no
tls_config	tls_config	Configure TLS settings for connecting to the endpoint.	no

The > symbol indicates deeper levels of nesting. For example, oauth2 > tls_config refers to a tls_config block defined inside an oauth2 block.

label_selector block

The label_selector block describes a Kubernetes label selector for rule or namespace discovery.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_labels`	`map(string)`	Label keys and values used to discover resources.	`{}`	yes

When the match_labels argument is empty, all resources will be matched.

match_expression block

The match_expression block describes a Kubernetes label match expression for rule or namespace discovery.

The following arguments are supported:

Name	Type	Description	Required
`key`	`string`	The label name to match against.	yes
`operator`	`string`	The operator to use when matching.	yes
`values`	`list(string)`	The values used when matching.	no

The operator argument should be one of the following strings:

"In"
"NotIn"
"Exists"
"DoesNotExist"

The values argument must not be provided when operator is set to "Exists" or "DoesNotExist".

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

authorization block

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

oauth2 block

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

tls_config block

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Exported fields

mimir.rules.kubernetes does not export any fields.

Component health

mimir.rules.kubernetes is reported as unhealthy if given an invalid configuration or an error occurs during reconciliation.

Debug information

mimir.rules.kubernetes exposes resource-level debug information.

The following are exposed per discovered PrometheusRule resource:

The Kubernetes namespace.
The resource name.
The resource uid.
The number of rule groups.

The following are exposed per discovered Mimir rule namespace resource:

The namespace name.
The number of rule groups.

Only resources managed by the component are exposed - regardless of how many actually exist.

Debug metrics

Metric Name	Type	Description
`mimir_rules_config_updates_total`	`counter`	Number of times the configuration has been updated.
`mimir_rules_events_total`	`counter`	Number of events processed, partitioned by event type.
`mimir_rules_events_failed_total`	`counter`	Number of events that failed to be processed, partitioned by event type.
`mimir_rules_events_retried_total`	`counter`	Number of events that were retried, partitioned by event type.
`mimir_rules_client_request_duration_seconds`	`histogram`	Duration of requests to the Mimir API.

Example

This example creates a mimir.rules.kubernetes component that loads discovered rules to a local Mimir instance under the team-a tenant. Only namespaces and rules with the agent label set to yes are included.

mimir.rules.kubernetes "local" {
    address = "mimir:8080"
    tenant_id = "team-a"

    rule_namespace_selector {
        match_labels = {
            agent = "yes",
        }
    }

    rule_selector {
        match_labels = {
            agent = "yes",
        }
    }
}

This example creates a mimir.rules.kubernetes component that loads discovered rules to Grafana Cloud.

mimir.rules.kubernetes "default" {
    address = "GRAFANA_CLOUD_METRICS_URL"
    basic_auth {
        username = "GRAFANA_CLOUD_USER"
        password = "GRAFANA_CLOUD_API_KEY"
        // Alternatively, load the password from a file:
        // password_file = "GRAFANA_CLOUD_API_KEY_PATH"
    }
}

The following example is an RBAC configuration for Kubernetes. It authorizes the Agent to query the Kubernetes REST API:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: grafana-agent
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: grafana-agent
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["monitoring.coreos.com"]
  resources: ["prometheusrules"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: grafana-agent
subjects:
- kind: ServiceAccount
  name: grafana-agent
  namespace: default
roleRef:
  kind: ClusterRole
  name: grafana-agent
  apiGroup: rbac.authorization.k8s.io

module.file

Fri, 10 Apr 2026 07:58:08 +0000

module.file (deprecated)

Caution
Starting with release v0.40, module.string is deprecated and is replaced by import.string. module.string will be removed in a future release.

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

module.file is a module loader component. A module loader is a Grafana Agent Flow component which retrieves a module and runs the components defined inside of it.

module.file simplifies the configurations for modules loaded from a file by embedding a local.file component. This allows a single module loader to do the equivalence of using the more generic module.string paired with a local.file component.

Usage

module.file "LABEL" {
  filename = FILENAME

  arguments {
    MODULE_ARGUMENT_1 = VALUE_1
    MODULE_ARGUMENT_2 = VALUE_2
    ...
  }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`filename`	`string`	Path of the file on disk to watch		yes
`detector`	`string`	Which file change detector to use (fsnotify, poll)	`"fsnotify"`	no
`poll_frequency`	`duration`	How often to poll for file changes	`"1m"`	no
`is_secret`	`bool`	Marks the file as containing a secret	`false`	no

File change detectors

File change detectors detect when the file needs to be re-read from disk. local.file supports two detectors: fsnotify and poll.

fsnotify

The component re-reads the watched file when a filesystem event is received. This re-read happens for any filesystem event related to the file, including a permissions change.

fsnotify also polls for changes to the file with the configured poll_frequency as a fallback.

fsnotify stops receiving filesystem events if the watched file has been deleted, renamed, or moved. The subscription is re-established on the next poll once the watched file exists again.

poll

The poll file change detector causes the watched file to be re-read every poll_frequency, regardless of whether the file changed.

Blocks

The following blocks are supported inside the definition of module.file:

Hierarchy	Block	Description	Required
arguments	arguments	Arguments to pass to the module.	no

arguments block

The arguments block specifies the list of values to pass to the loaded module.

The attributes provided in the arguments block are validated based on the argument blocks defined in the module source:

If a module source marks one of its arguments as required, it must be provided as an attribute in the arguments block of the module loader.
Attributes in the argument block of the module loader will be rejected if they are not defined in the module source.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`exports`	`map(any)`	The exports of the Module loader.

exports exposes the export config block inside a module. It can be accessed from the parent config via module.file.LABEL.exports.EXPORT_LABEL.

Values in exports correspond to export blocks defined in the module source.

Component health

module.file is reported as healthy if the most recent load of the module was successful.

If the module is not loaded successfully, the current health displays as unhealthy and the health includes the error from loading the module.

Debug information

module.file does not expose any component-specific debug information.

Debug metrics

module.file does not expose any component-specific debug metrics.

Example

In this example, we pass credentials from a parent config to a module which loads a prometheus.remote_write component. The exports of the prometheus.remote_write component are exposed to parent config, allowing the parent config to pass metrics to it.

Parent:

module.file "metrics" {
  filename = "/path/to/prometheus_remote_write_module.river"

  arguments {
    username = env("PROMETHEUS_USERNAME")
    password = env("PROMETHEUS_PASSWORD")
  }
}

prometheus.exporter.unix "default" { }

prometheus.scrape "local_agent" {
  targets         = prometheus.exporter.unix.default.targets
  forward_to      = [module.file.metrics.exports.prometheus_remote_write.receiver]
  scrape_interval = "10s"
}

Module:

argument "username" { }

argument "password" { }

export "prometheus_remote_write" {
  value = prometheus.remote_write.grafana_cloud
}

prometheus.remote_write "grafana_cloud" {
  endpoint {
    url = "https://prometheus-us-central1.grafana.net/api/prom/push"

    basic_auth {
      username = argument.username.value
      password = argument.password.value
    }
  }
}

module.git

Fri, 10 Apr 2026 07:58:08 +0000

module.git (deprecated)

Caution
Starting with release v0.40, module.git is deprecated and is replaced by import.git. module.git will be removed in a future release.

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

module.git is a module loader component. A module loader is a Grafana Agent Flow component which retrieves a module and runs the components defined inside of it.

module.git retrieves a module source from a file in a Git repository.

Usage

module.git "LABEL" {
  repository = "GIT_REPOSTORY"
  path       = "PATH_TO_MODULE"

  arguments {
    MODULE_ARGUMENT_1 = VALUE_1
    MODULE_ARGUMENT_2 = VALUE_2
    ...
  }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`repository`	`string`	The Git repository address to retrieve the module from.		yes
`revision`	`string`	The Git revision to retrieve the module from.	`"HEAD"`	no
`path`	`string`	The path in the repository where the module is stored.		yes
`pull_frequency`	`duration`	The frequency to pull the repository for updates.	`"60s"`	no

The repository attribute must be set to a repository address that would be recognized by Git with a git clone REPOSITORY_ADDRESS command, such as htts://github.com/grafana/agent.git.

The revision attribute, when provided, must be set to a valid branch, tag, or commit SHA within the repository.

The path attribute must be set to a path which is accessible from the root of the repository, such as FILE_NAME.river or FOLDER_NAME/FILE_NAME.river.

If pull_frequency is not "0s", the Git repository will be pulled for updates at the frequency specified, causing the loaded module to update with the retrieved changes.

Blocks

The following blocks are supported inside the definition of module.git:

Hierarchy	Block	Description	Required
basic_auth	basic_auth	Configure basic_auth for authenticating to the repo.	no
ssh_key	ssh_key	Configure a SSH Key for authenticating to the repo.	no
arguments	arguments	Arguments to pass to the module.	no

basic_auth block

Name	Type	Description	Required
`password_file`	`string`	File containing the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

ssh_key block

Name	Type	Description	Required
`username`	`string`	SSH username.	yes
`key`	`secret`	SSH private key	no
`key_file`	`string`	SSH private key path.	no
`passphrase`	`secret`	Passphrase for SSH key if needed.	no

arguments block

The arguments block specifies the list of values to pass to the loaded module.

The attributes provided in the arguments block are validated based on the argument blocks defined in the module source:

If a module source marks one of its arguments as required, it must be provided as an attribute in the arguments block of the module loader.
Attributes in the argument block of the module loader will be rejected if they are not defined in the module source.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`exports`	`map(any)`	The exports of the Module loader.

exports exposes the export config block inside a module. It can be accessed from the parent config via module.git.COMPONENT_LABEL.exports.EXPORT_LABEL.

Values in exports correspond to export blocks defined in the module source.

Component health

module.git is reported as healthy if the repository was cloned successfully and most recent load of the module was successful.

Debug information

module.git includes debug information for:

The full SHA of the currently checked out revision.
The most recent error when trying to fetch the repository, if any.

Debug metrics

module.git does not expose any component-specific debug metrics.

Examples

This example uses a module loaded from a Git repository which adds two numbers:

module.git "add" {
  repository = "https://github.com/rfratto/agent-modules.git"
  revision   = "main"
  path       = "add/module.river"

  arguments {
    a = 15
    b = 45
  }
}

The same example as above using basic auth:

module.git "add" {
  repository = "https://github.com/rfratto/agent-modules.git"
  revision   = "main"
  path       = "add/module.river"

  basic_auth {
    username = "USERNAME"
    password = "PASSWORD"
  }

  arguments {
    a = 15
    b = 45
  }
}

Using SSH Key from another component:

local.file "ssh_key" {
  filename = "PATH/TO/SSH.KEY"
  is_secret = true
}

module.git "add" {
  repository = "github.com:rfratto/agent-modules.git"
  revision   = "main"
  path       = "add/module.river"

  ssh_key {
    username = "git"
    key = local.file.ssh_key.content
  }

  arguments {
    a = 15
    b = 45
  }
}

The same example as above using SSH Key auth:

module.git "add" {
  repository = "github.com:rfratto/agent-modules.git"
  revision   = "main"
  path       = "add/module.river"

  ssh_key {
    username = "git"
    key_file = "PATH/TO/SSH.KEY"
  }

  arguments {
    a = 15
    b = 45
  }
}

module.http

Fri, 10 Apr 2026 07:58:08 +0000

module.http (deprecated)

Caution
Starting with release v0.40, module.http is deprecated and is replaced by import.http. module.http will be removed in a future release.

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

module.http is a module loader component.

module.http embeds a remote.http component to retrieve the module from a remote HTTP server. This allows you to use a single module loader, rather than a remote.http component paired with a module.string component.

Usage

module.http "LABEL" {
  url = URL

  arguments {
    MODULE_ARGUMENT_1 = VALUE_1
    ...
  }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`url`	`string`	URL to poll.		yes
`method`	`string`	Define HTTP method for the request	`"GET"`	no
`headers`	`map(string)`	Custom headers for the request.	`{}`	no
`poll_frequency`	`duration`	Frequency to poll the URL.	`"1m"`	no
`poll_timeout`	`duration`	Timeout when polling the URL.	`"10s"`	no
`is_secret`	`bool`	Whether the response body should be treated as a secret.	false	no

Blocks

The following blocks are supported inside the definition of module.http:

Hierarchy	Block	Description	Required
arguments	arguments	Arguments to pass to the module.	no

arguments block

The arguments block specifies the list of values to pass to the loaded module.

The attributes provided in the arguments block are validated based on the argument blocks defined in the module source:

If a module source marks one of its arguments as required, it must be provided as an attribute in the arguments block of the module loader.
Attributes in the argument block of the module loader are rejected if they are not defined in the module source.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`exports`	`map(any)`	The exports of the Module loader.

exports exposes the export config block inside a module. It can be accessed from the parent config via module.http.LABEL.exports.EXPORT_LABEL.

Values in exports correspond to export blocks defined in the module source.

Component health

module.http is reported as healthy if the most recent load of the module was successful.

Before the first load of the module, the health is reported as Unknown.

If the module is not loaded successfully, the current health displays as unhealthy, and the health includes the error from loading the module.

Debug information

module.http does not expose any component-specific debug information.

Debug metrics

module.http does not expose any component-specific debug metrics.

Example

In this example, the module.http component loads a module from a locally running HTTP server, polling for changes once every minute.

The module sets up a Redis exporter and exports the list of targets to the parent config to scrape and remote write.

Parent:

module.http "remote_module" {
  url              = "http://localhost:8080/redis_module.yaml"
  poll_frequency   = "1m"
}

prometheus.exporter.unix "default" { }

prometheus.scrape "local_agent" {
  targets         = concat(prometheus.exporter.unix.default.targets, module.http.remote_module.exports.targets)
  forward_to      = [module.http.metrics.exports.prometheus_remote_write.receiver]
  scrape_interval = "10s"
}

prometheus.remote_write "default" {
  endpoint {
    url = PROMETHEUS_REMOTE_WRITE_URL

    basic_auth {
        username = USERNAME
        password = PASSWORD
    }
  }
}

Replace the following:

PROMETHEUS_REMOTE_WRITE_URL: The URL of the Prometheus remote_write-compatible server to send metrics to.
USERNAME: The username to use for authentication to the remote_write API.
PASSWORD: The password to use for authentication to the remote_write API.

Module:

prometheus.exporter.redis "local_redis" {
  redis_addr          = REDIS_ADDR
  redis_password_file = REDIS_PASSWORD_FILE
}

export "redis_targets" {
  value = prometheus.exporter.redis.local_redis.targets
}

Replace the following:

REDIS_ADDR: The address of your Redis instance.
REDIS_PASSWORD_FILE: The path to a file containing the password for your Redis instance.

module.string

Fri, 10 Apr 2026 07:58:08 +0000

module.string (deprecated)

Caution
Starting with release v0.40, module.string is deprecated and is replaced by import.string. module.string will be removed in a future release.

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

module.string is a module loader component. A module loader is a Grafana Agent Flow component which retrieves a module and runs the components defined inside of it.

Usage

module.string "LABEL" {
  content = CONTENT

  arguments {
    MODULE_ARGUMENT_1 = VALUE_1
    MODULE_ARGUMENT_2 = VALUE_2
    ...
  }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`content`	`secret` or `string`	The contents of the module to load as a secret or string.		yes

content is a string that contains the configuration of the module to load. content is typically loaded by using the exports of another component. For example,

local.file.LABEL.content
remote.http.LABEL.content
remote.s3.LABEL.content

Blocks

The following blocks are supported inside the definition of module.string:

Hierarchy	Block	Description	Required
arguments	arguments	Arguments to pass to the module.	no

arguments block

The arguments block specifies the list of values to pass to the loaded module.

The attributes provided in the arguments block are validated based on the argument blocks defined in the module source:

If a module source marks one of its arguments as required, it must be provided as an attribute in the arguments block of the module loader.
Attributes in the argument block of the module loader will be rejected if they are not defined in the module source.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`exports`	`map(any)`	The exports of the Module loader.

exports exposes the export config block inside a module. It can be accessed from the parent config via module.string.LABEL.exports.EXPORT_LABEL.

Values in exports correspond to export blocks defined in the module source.

Component health

module.string is reported as healthy if the most recent load of the module was successful.

If the module is not loaded successfully, the current health displays as unhealthy and the health includes the error from loading the module.

Debug information

module.string does not expose any component-specific debug information.

Debug metrics

module.string does not expose any component-specific debug metrics.

Example

Parent:

local.file "metrics" {
  filename = "/path/to/prometheus_remote_write_module.river"
}

module.string "metrics" {
  content = local.file.metrics.content

  arguments {
    username = env("PROMETHEUS_USERNAME")
    password = env("PROMETHEUS_PASSWORD")
  }
}

prometheus.exporter.unix "default" { }

prometheus.scrape "local_agent" {
  targets         = prometheus.exporter.unix.default.targets
  forward_to      = [module.string.metrics.exports.prometheus_remote_write.receiver]
  scrape_interval = "10s"
}

Module:

argument "username" { }

argument "password" { }

export "prometheus_remote_write" {
  value = prometheus.remote_write.grafana_cloud
}

prometheus.remote_write "grafana_cloud" {
  endpoint {
    url = "https://prometheus-us-central1.grafana.net/api/prom/push"

    basic_auth {
      username = argument.username.value
      password = argument.password.value
    }
  }
}

otelcol.auth.basic

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.auth.basic

otelcol.auth.basic exposes a handler that can be used by other otelcol components to authenticate requests using basic authentication.

This extension supports both server and client authentication.

NOTE: otelcol.auth.basic is a wrapper over the upstream OpenTelemetry Collector basicauth extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.auth.basic components can be specified by giving them different labels.

Usage

otelcol.auth.basic "LABEL" {
  username = "USERNAME"
  password = "PASSWORD"
}

Arguments

otelcol.auth.basic supports the following arguments:

Name	Type	Description	Default	Required
`username`	`string`	Username to use for basic authentication requests.		yes
`password`	`secret`	Password to use for basic authentication requests.		yes

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`handler`	`capsule(otelcol.Handler)`	A value that other components can use to authenticate requests.

Component health

otelcol.auth.basic is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.auth.basic does not expose any component-specific debug information.

Example

This example configures otelcol.exporter.otlp to use basic authentication:

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.basic.creds.handler
  }
}

otelcol.auth.basic "creds" {
  username = "demo"
  password = env("API_KEY")
}

otelcol.auth.bearer

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.auth.bearer

otelcol.auth.bearer exposes a handler that can be used by other otelcol components to authenticate requests using bearer token authentication.

This extension supports both server and client authentication.

NOTE: otelcol.auth.bearer is a wrapper over the upstream OpenTelemetry Collector bearertokenauth extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.auth.bearer components can be specified by giving them different labels.

Usage

otelcol.auth.bearer "LABEL" {
  token = "TOKEN"
}

Arguments

otelcol.auth.bearer supports the following arguments:

Name	Type	Description	Default	Required
`token`	`secret`	Bearer token to use for authenticating requests.		yes
`scheme`	`string`	Authentication scheme name.	“Bearer”	no

When sending the token, the value of scheme is prepended to the token value. The string is then sent out as either a header (in case of HTTP) or as metadata (in case of gRPC).

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`handler`	`capsule(otelcol.Handler)`	A value that other components can use to authenticate requests.

Component health

otelcol.auth.bearer is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.auth.bearer does not expose any component-specific debug information.

Examples

Default scheme via gRPC transport

The example below configures otelcol.exporter.otlp to use a bearer token authentication.

If we assume that the value of the API_KEY environment variable is SECRET_API_KEY, then the Authorization RPC metadata is set to Bearer SECRET_API_KEY.

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.bearer.creds.handler
  }
}

otelcol.auth.bearer "creds" {
  token = env("API_KEY")
}

Custom scheme via HTTP transport

The example below configures otelcol.exporter.otlphttp to use a bearer token authentication.

If we assume that the value of the API_KEY environment variable is SECRET_API_KEY, then the Authorization HTTP header is set to MyScheme SECRET_API_KEY.

otelcol.exporter.otlphttp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.bearer.creds.handler
  }
}

otelcol.auth.bearer "creds" {
  token = env("API_KEY")
  scheme = "MyScheme"
}

otelcol.auth.headers

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.auth.headers

otelcol.auth.headers exposes a handler that can be used by other otelcol components to authenticate requests using custom headers.

NOTE: otelcol.auth.headers is a wrapper over the upstream OpenTelemetry Collector headerssetter extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.auth.headers components can be specified by giving them different labels.

Usage

otelcol.auth.headers "LABEL" {
  header {
    key   = "HEADER_NAME"
    value = "HEADER_VALUE"
  }
}

Arguments

otelcol.auth.headers doesn’t support any arguments and is configured fully through inner blocks.

Blocks

The following blocks are supported inside the definition of otelcol.auth.headers:

Hierarchy	Block	Description	Required
header	header	Custom header to attach to requests.	no

header block

The header block defines a custom header to attach to requests. It is valid to provide multiple header blocks to set more than one header.

Name	Type	Description	Default	Required
`key`	`string`	Name of the header to set.		yes
`value`	`string` or `secret`	Value of the header.		no
`from_context`	`string`	Metadata name to get header value from.		no
`action`	`string`	An action to perform on the header	“upsert”	no

The supported values for action are:

insert: Inserts the new header if it does not exist.
update: Updates the header value if it exists.
upsert: Inserts a header if it does not exist and updates the header if it exists.
delete: Deletes the header.

Exactly one of value or from_context must be provided for each header block.

The value attribute sets the value of the header directly. Alternatively, from_context can be used to dynamically retrieve the header value from request metadata.

For from_context to work, other components in the pipeline also need to be configured appropriately:

If an otelcol.processor.batch is present in the pipeline, it must be configured to preserve client metadata. Do this by adding the value that from_context needs to the metadata_keys of the batch processor.
otelcol receivers must be configured with include_metadata set to true so that metadata keys are available to the pipeline.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`handler`	`capsule(otelcol.Handler)`	A value that other components can use to authenticate requests.

Component health

otelcol.auth.headers is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.auth.headers does not expose any component-specific debug information.

Example

This example configures otelcol.exporter.otlp to use custom headers:

otelcol.receiver.otlp "default" {
  http {
    include_metadata = true
  }
  grpc {
    include_metadata = true
  }

  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  // Preserve the tenant_id metadata.
  metadata_keys = ["tenant_id"]

  output {
    metrics = [otelcol.exporter.otlp.production.input]
    logs    = [otelcol.exporter.otlp.production.input]
    traces  = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.auth.headers "creds" {
  header {
    key          = "X-Scope-OrgID"
    from_context = "tenant_id"
  }

  header {
    key   = "User-ID"
    value = "user_id"
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
    auth     = otelcol.auth.headers.creds.handler
  }
}

otelcol.auth.oauth2

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.auth.oauth2

otelcol.auth.oauth2 exposes a handler that can be used by other otelcol components to authenticate requests using OAuth 2.0.

The authorization tokens can be used by HTTP and gRPC based OpenTelemetry exporters. This component can fetch and refresh expired tokens automatically. For further details about OAuth 2.0 Client Credentials flow (2-legged workflow) see this document.

NOTE: otelcol.auth.oauth2 is a wrapper over the upstream OpenTelemetry Collector oauth2client extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.auth.oauth2 components can be specified by giving them different labels.

Usage

otelcol.auth.oauth2 "LABEL" {
    client_id     = "CLIENT_ID"
    client_secret = "CLIENT_SECRET"
    token_url     = "TOKEN_URL"
}

Arguments

Name	Type	Description	Default	Required
`client_id`	`string`	The client identifier issued to the client.		no
`client_id_file`	`string`	The file path to retrieve the client identifier issued to the client.		no
`client_secret`	`secret`	The secret string associated with the client identifier.		no
`client_secret_file`	`secret`	The file path to retrieve the secret string associated with the client identifier.		no
`token_url`	`string`	The server endpoint URL from which to get tokens.		yes
`endpoint_params`	`map(list(string))`	Additional parameters that are sent to the token endpoint.	`{}`	no
`scopes`	`list(string)`	Requested permissions associated for the client.	`[]`	no
`timeout`	`duration`	The timeout on the client connecting to `token_url`.	`"0s"`	no

The timeout argument is used both for requesting initial tokens and for refreshing tokens. "0s" implies no timeout.

At least one of the client_id and client_id_file pair of arguments must be set. In case both are set, client_id_file takes precedence.

Similarly, at least one of the client_secret and client_secret_file pair of arguments must be set. In case both are set, client_secret_file also takes precedence.

Blocks

The following blocks are supported inside the definition of otelcol.auth.oauth2:

Hierarchy	Block	Description	Required
tls	tls	TLS settings for the token client.	no

tls block

The tls block configures TLS settings used for connecting to the token client. If the tls block isn’t provided, TLS won’t be used for communication.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`handler`	`capsule(otelcol.Handler)`	A value that other components can use to authenticate requests.

Component health

otelcol.auth.oauth2 is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.auth.oauth2 does not expose any component-specific debug information.

Example

This example configures otelcol.exporter.otlp to use OAuth 2.0 for authentication:

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.oauth2.creds.handler
  }
}

otelcol.auth.oauth2 "creds" {
    client_id     = "someclientid"
    client_secret = "someclientsecret"
    token_url     = "https://example.com/oauth2/default/v1/token"
}

Here is another example with some optional attributes specified:

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.oauth2.creds.handler
  }
}

otelcol.auth.oauth2 "creds" {
    client_id       = "someclientid2"
    client_secret   = "someclientsecret2"
    token_url       = "https://example.com/oauth2/default/v1/token"
    endpoint_params = {"audience" = ["someaudience"]}
    scopes          = ["api.metrics"]
    timeout         = "3600s"
}

otelcol.auth.sigv4

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.auth.sigv4

otelcol.auth.sigv4 exposes a handler that can be used by other otelcol components to authenticate requests to AWS services using the AWS Signature Version 4 (SigV4) protocol. For more information about SigV4 see the AWS documentation about Signing AWS API requests .

NOTE: otelcol.auth.sigv4 is a wrapper over the upstream OpenTelemetry Collector sigv4auth extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.auth.sigv4 components can be specified by giving them different labels.

NOTE: The Agent must have valid AWS credentials as used by the AWS SDK for Go.

Usage

otelcol.auth.sigv4 "LABEL" {
}

Arguments

Name	Type	Description	Default	Required
`region`	`string`	The AWS region to sign with.	""	no
`service`	`string`	The AWS service to sign with.	""	no

If region and service are left empty, their values are inferred from the URL of the exporter using the following rules:

If the exporter URL starts with aps-workspaces and service is empty, service will be set to aps.
If the exporter URL starts with search- and service is empty, service will be set to es.
If the exporter URL starts with either aps-workspaces or search- and region is empty, region . will be set to the value between the first and second . character in the exporter URL.

If none of the above rules apply, then region and service must be specified.

A list of valid AWS regions can be found on Amazon’s documentation for Regions, Availability Zones, and Local Zones.

Blocks

The following blocks are supported inside the definition of otelcol.auth.sigv4:

Hierarchy	Block	Description	Required
assume_role	assume_role	Configuration for assuming a role.	no

assume_role block

The assume_role block specifies the configuration needed to assume a role.

Name	Type	Description	Default	Required
`arn`	`string`	The Amazon Resource Name (ARN) of a role to assume.	""	no
`session_name`	`string`	The name of a role session.	""	no
`sts_region`	`string`	The AWS region where STS is used to assume the configured role.	""	no

If the assume_role block is specified in the config and sts_region is not set, then sts_region will default to the value for region.

For cross region authentication, region and sts_region can be set different to different values.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`handler`	`capsule(otelcol.Handler)`	A value that other components can use to authenticate requests.

Component health

otelcol.auth.sigv4 is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.auth.sigv4 does not expose any component-specific debug information.

Examples

Inferring the “region” and “service” from an “aps-workspaces” exporter endpoint

In this example the exporter endpoint starts with aps-workspaces. Hence service is inferred to be aps and region is inferred to be us-east-1.

otelcol.exporter.otlp "example" {
  client {
    endpoint = "https://aps-workspaces.us-east-1.amazonaws.com/workspaces/ws-XXX/api/v1/remote_write"
    auth     = otelcol.auth.sigv4.creds.handler
  }
}

otelcol.auth.sigv4 "creds" {
}

Inferring the “region” and “service” from a “search-” exporter endpoint

In this example the exporter endpoint starts with search-. Hence service is inferred to be es and region is inferred to be us-east-1.

otelcol.exporter.otlp "example" {
  client {
    endpoint = "https://search-my-domain.us-east-1.es.amazonaws.com/_search?q=house"
    auth     = otelcol.auth.sigv4.creds.handler
  }
}

otelcol.auth.sigv4 "creds" {
}

Specifying “region” and “service” explicitly

In this example the exporter endpoint does not begin with search- or with aps-workspaces. Hence, we need to specify region and service explicitly.

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.sigv4.creds.handler
  }
}

otelcol.auth.sigv4 "creds" {
    region = "example_region"
    service = "example_service"
}

Specifying “region” and “service” explicitly and adding a “role” to assume

In this example we have also specified configuration to assume a role. sts_region has not been provided, so it will default to the value of region which is example_region.

otelcol.exporter.otlp "example" {
  client {
    endpoint = "my-otlp-grpc-server:4317"
    auth     = otelcol.auth.sigv4.creds.handler
  }
}

otelcol.auth.sigv4 "creds" {
  region  = "example_region"
  service = "example_service"
  
  assume_role {
    session_name = "role_session_name"
  }
}

otelcol.connector.host_info

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.connector.host_info

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otel.connector.host_info accepts span data from other otelcol components and generates usage metrics.

Usage

otelcol.connector.host_info "LABEL" {
  output {
    metrics = [...]
  }
}

Arguments

otelcol.connector.host_info supports the following arguments:

Name	Type	Description	Default	Required
`host_identifiers`	`list(string)`	Ordered list of resource attributes used to identify unique hosts.	`["host.id"]`	no
`metrics_flush_interval`	`duration`	How often to flush generated metrics.	`"60s"`	no

Blocks

The following blocks are supported inside the definition of otelcol.connector.host_info:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the metrics argument accordingly to send telemetry data to other components.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer traces telemetry data. It does not accept metrics and logs.

Example

The example below accepts traces, adds the host.id resource attribute via the otelcol.processor.resourcedetection component, creates usage metrics from these traces, and writes the metrics to Mimir.

otelcol.receiver.otlp "otlp" {
  http {}
  grpc {}

  output {
    traces = [otelcol.processor.resourcedetection.otlp_resources.input]
  }
}

otelcol.processor.resourcedetection "otlp_resources" {
  detectors = ["system"]
  system {
    hostname_sources = [ "os" ]
    resource_attributes {
      host.id {
        enabled = true
      }
    }
  }
  output {
    traces = [otelcol.connector.host_info.default.input]
  }
}

otelcol.connector.host_info "default" {
  output {
    metrics = [otelcol.exporter.prometheus.otlp_metrics.input]
  }
}

otelcol.exporter.prometheus "otlp_metrics" {
  forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
  endpoint {
    url = "https://prometheus-xxx.grafana.net/api/prom/push"
    basic_auth {
      username = env("PROMETHEUS_USERNAME")
      password = env("GRAFANA_CLOUD_API_KEY")
    }
  }
}

Compatible components

otelcol.connector.host_info can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.connector.host_info has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.connector.servicegraph

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.connector.servicegraph

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.connector.servicegraph accepts span data from other otelcol components and outputs metrics representing the relationship between various services in a system. A metric represents an edge in the service graph. Those metrics can then be used by a data visualization application (e.g. Grafana) to draw the service graph.

NOTE: otelcol.connector.servicegraph is a wrapper over the upstream OpenTelemetry Collector servicegraph connector. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.connector.servicegraph components can be specified by giving them different labels.

This component is based on Grafana Tempo’s service graph processor.

Service graphs are useful for a number of use-cases:

Infer the topology of a distributed system. As distributed systems grow, they become more complex. Service graphs can help you understand the structure of the system.
Provide a high level overview of the health of your system. Service graphs show error rates, latencies, and other relevant data.
Provide a historic view of a system’s topology. Distributed systems change very frequently, and service graphs offer a way of seeing how these systems have evolved over time.

Since otelcol.connector.servicegraph has to process both sides of an edge, it needs to process all spans of a trace to function properly. If spans of a trace are spread out over multiple Agent instances, spans cannot be paired reliably. A solution to this problem is using otelcol.exporter.loadbalancing in front of Agent instances running otelcol.connector.servicegraph.

Usage

otelcol.connector.servicegraph "LABEL" {
  output {
    metrics = [...]
  }
}

Arguments

otelcol.connector.servicegraph supports the following arguments:

Name	Type	Description	Default	Required
`latency_histogram_buckets`	`list(duration)`	Buckets for latency histogram metrics.	`["2ms", "4ms", "6ms", "8ms", "10ms", "50ms", "100ms", "200ms", "400ms", "800ms", "1s", "1400ms", "2s", "5s", "10s", "15s"]`	no
`dimensions`	`list(string)`	A list of dimensions to add with the default dimensions.	`[]`	no
`cache_loop`	`duration`	Configures how often to delete series which have not been updated.	`"1m"`	no
`store_expiration_loop`	`duration`	The time to expire old entries from the store periodically.	`"2s"`	no
`metrics_flush_interval`	`duration`	The interval at which metrics are flushed to downstream components.	`"0s"`	no

Service graphs work by inspecting traces and looking for spans with parent-children relationship that represent a request. otelcol.connector.servicegraph uses OpenTelemetry semantic conventions to detect a myriad of requests. The following requests are currently supported:

A direct request between two services, where the outgoing and the incoming span must have a Span Kind value of client and server respectively.
A request across a messaging system, where the outgoing and the incoming span must have a Span Kind value of producer and consumer respectively.
A database request, where spans have a Span Kind with a value of client, as well as an attribute with a key of db.name.

Every span which can be paired up to form a request is kept in an in-memory store:

If the TTL of the span expires before it can be paired, it is deleted from the store. TTL is configured in the store block.
If the span is paired prior to its expiration, a metric is recorded and the span is deleted from the store.

The following metrics are emitted by the processor:

Metric	Type	Labels	Description
traces_service_graph_request_total	Counter	client, server, connection_type	Total count of requests between two nodes
traces_service_graph_request_failed_total	Counter	client, server, connection_type	Total count of failed requests between two nodes
traces_service_graph_request_server_seconds	Histogram	client, server, connection_type	Time for a request between two nodes as seen from the server
traces_service_graph_request_client_seconds	Histogram	client, server, connection_type	Time for a request between two nodes as seen from the client
traces_service_graph_unpaired_spans_total	Counter	client, server, connection_type	Total count of unpaired spans
traces_service_graph_dropped_spans_total	Counter	client, server, connection_type	Total count of dropped spans

Duration is measured both from the client and the server sides.

The latency_histogram_buckets argument controls the buckets for traces_service_graph_request_server_seconds and traces_service_graph_request_client_seconds.

Each emitted metrics series have a client and a server label corresponding with the service doing the request and the service receiving the request. The value of the label is derived from the service.name resource attribute of the two spans.

The connection_type label may not be set. If it is set, its value will be either messaging_system or database.

Additional labels can be included using the dimensions configuration option:

Those labels will have a prefix to mark where they originate (client or server span kinds). The client_ prefix relates to the dimensions coming from spans with a Span Kind of client. The server_ prefix relates to the dimensions coming from spans with a Span Kind of server.
Firstly the resource attributes will be searched. If the attribute is not found, the span attributes will be searched.

When metrics_flush_interval is set to 0s, metrics will be flushed on every received batch of traces.

Blocks

The following blocks are supported inside the definition of otelcol.connector.servicegraph:

Hierarchy	Block	Description	Required
store	store	Configures the in-memory store for spans.	no
output	output	Configures where to send telemetry data.	yes

store block

The store block configures the in-memory store for spans.

Name	Type	Description	Default	Required
`max_items`	`number`	Maximum number of items to keep in the store.	`1000`	no
`ttl`	`duration`	The time to live for spans in the store.	`"2s"`	no

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer traces telemetry data. It does not accept metrics and logs.

Component health

otelcol.connector.servicegraph is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.connector.servicegraph does not expose any component-specific debug information.

Example

The example below accepts traces, creates service graph metrics from them, and writes the metrics to Mimir. The traces are written to Tempo.

otelcol.connector.servicegraph also adds a label to each metric with the value of the “http.method” span/resource attribute.

otelcol.receiver.otlp "default" {
  grpc {
    endpoint = "0.0.0.0:4320"
  }
  
  output {
    traces  = [otelcol.connector.servicegraph.default.input,otelcol.exporter.otlp.grafana_cloud_tempo.input]
  }
}

otelcol.connector.servicegraph "default" {
  dimensions = ["http.method"]
  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "https://prometheus-xxx.grafana.net/api/prom/push"
    
    basic_auth {
      username = env("PROMETHEUS_USERNAME")
      password = env("GRAFANA_CLOUD_API_KEY")
    }
  }
}

otelcol.exporter.otlp "grafana_cloud_tempo" {
  client {
    endpoint = "https://tempo-xxx.grafana.net/tempo"
    auth     = otelcol.auth.basic.grafana_cloud_tempo.handler
  }
}

otelcol.auth.basic "grafana_cloud_tempo" {
  username = env("TEMPO_USERNAME")
  password = env("GRAFANA_CLOUD_API_KEY")
}

Some of the metrics in Mimir may look like this:

traces_service_graph_request_total{client="shop-backend",failed="false",server="article-service",client_http_method="DELETE",server_http_method="DELETE"}
traces_service_graph_request_failed_total{client="shop-backend",client_http_method="POST",failed="false",server="auth-service",server_http_method="POST"}
```<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`otelcol.connector.servicegraph` can accept arguments from the following components:

- Components that export [OpenTelemetry `otelcol.Consumer`](../../compatibility/#opentelemetry-otelcolconsumer-exporters)

`otelcol.connector.servicegraph` has exports that can be consumed by the following components:

- Components that consume [OpenTelemetry `otelcol.Consumer`](../../compatibility/#opentelemetry-otelcolconsumer-consumers)



Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.


<!-- END GENERATED COMPATIBLE COMPONENTS -->

otelcol.connector.spanlogs

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.connector.spanlogs

otelcol.connector.spanlogs accepts traces telemetry data from other otelcol components and outputs logs telemetry data for each span, root, or process. This allows you to automatically build a mechanism for trace discovery.

NOTE: otelcol.connector.spanlogs is a custom component unrelated to any components from the OpenTelemetry Collector. It is based on the automatic_logging component in the traces subsystem of the Agent static mode.

You can specify multiple otelcol.connector.spanlogs components by giving them different labels.

Usage

otelcol.connector.spanlogs "LABEL" {
  output {
    logs    = [...]
  }
}

Arguments

otelcol.connector.spanlogs supports the following arguments:

Name	Type	Description	Default	Required
`spans`	`bool`	Log one line per span.	`false`	no
`roots`	`bool`	Log one line for every root span of a trace.	`false`	no
`processes`	`bool`	Log one line for every process.	`false`	no
`span_attributes`	`list(string)`	Additional span attributes to log.	`[]`	no
`process_attributes`	`list(string)`	Additional process attributes to log.	`[]`	no
`labels`	`list(string)`	A list of keys that will be logged as labels.	`[]`	no

The values listed in labels should be the values of either span or process attributes.

WARNING: Setting spans to true could lead to a high volume of logs.

Blocks

The following blocks are supported inside the definition of otelcol.connector.spanlogs:

Hierarchy	Block	Description	Required
overrides	overrides	Overrides for keys in the log body.	no
output	output	Configures where to send received telemetry data.	yes

overrides block

The overrides block configures overrides for keys that will be logged in the body of the log line.

The following attributes are supported:

Name	Type	Description	Default	Required
`logs_instance_tag`	`string`	Indicates if the log line is for a span, root, or process.	`traces`	no
`service_key`	`string`	Log key for the service name of the resource.	`svc`	no
`span_name_key`	`string`	Log key for the name of the span.	`span`	no
`status_key`	`string`	Log key for the status of the span.	`status`	no
`duration_key`	`string`	Log key for the duration of the span.	`dur`	no
`trace_id_key`	`string`	Log key for the trace ID of the span.	`tid`	no

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the logs argument accordingly to send telemetry data to other components, .

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.connector.spanlogs is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.connector.spanlogs does not expose any component-specific debug information.

Example

The following configuration sends logs derived from spans to Loki.

Additionally, otelcol.processor.attributes is configured with a “hint” so that otelcol.exporter.loki promotes the span’s “attribute1” attribute to a Loki label.

otelcol.receiver.otlp "default" {
  grpc {}

  output {
    traces = [otelcol.connector.spanlogs.default.input]
  }
}

otelcol.connector.spanlogs "default" {
  spans              = true
  roots              = true
  processes          = true
  labels             = ["attribute1", "res_attribute1"]
  span_attributes    = ["attribute1"]
  process_attributes = ["res_attribute1"]

  output {
    logs = [otelcol.processor.attributes.default.input]
  }
}

otelcol.processor.attributes "default" {
  action {
    key = "loki.attribute.labels"
    action = "insert"
    value = "attribute1"
  }

  output {
    logs = [otelcol.exporter.loki.default.input]
  }
}

otelcol.exporter.loki "default" {
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100"
  }
}

For an input trace like this…

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "res_attribute1",
            "value": { "intValue": "78" }
          },
          {
            "key": "unused_res_attribute1",
            "value": { "stringValue": "str" }
          },
          {
            "key": "res_account_id",
            "value": { "intValue": "2245" }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "trace_id": "7bba9f33312b3dbb8b2c2c62bb7abe2d",
              "span_id": "086e83747d0e381e",
              "name": "TestSpan",
              "attributes": [
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                },
                {
                  "key": "unused_attribute1",
                  "value": { "intValue": "78" }
                },
                {
                  "key": "account_id",
                  "value": { "intValue": "2245" }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

… the output log coming out of otelcol.connector.spanlogs will look like this:

{
  "resourceLogs": [
    {
      "scopeLogs": [
        {
          "log_records": [
            {
              "body": {
                "stringValue": "span=TestSpan dur=0ns attribute1=78 svc=TestSvcName res_attribute1=78 tid=7bba9f33312b3dbb8b2c2c62bb7abe2d"
              },
              "attributes": [
                {
                  "key": "traces",
                  "value": { "stringValue": "span" }
                },
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                },
                {
                  "key": "res_attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            },
            {
              "body": {
                "stringValue": "span=TestSpan dur=0ns attribute1=78 svc=TestSvcName res_attribute1=78 tid=7bba9f33312b3dbb8b2c2c62bb7abe2d"
              },
              "attributes": [
                {
                  "key": "traces",
                  "value": { "stringValue": "root" }
                },
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                },
                {
                  "key": "res_attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            },
            {
              "body": {
                "stringValue": "svc=TestSvcName res_attribute1=78 tid=7bba9f33312b3dbb8b2c2c62bb7abe2d"
              },
              "attributes": [
                {
                  "key": "traces",
                  "value": { "stringValue": "process" }
                },
                {
                  "key": "res_attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Compatible components

otelcol.connector.spanlogs can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.connector.spanlogs has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.connector.spanmetrics

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.connector.spanmetrics

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.connector.spanmetrics accepts span data from other otelcol components and aggregates Request, Error and Duration (R.E.D) OpenTelemetry metrics from the spans:

Request counts are computed as the number of spans seen per unique set of dimensions, including Errors. Multiple metrics can be aggregated if, for instance, a user wishes to view call counts just on service.name and span.name.

Requests are tracked using a calls metric with a status.code datapoint attribute set to Ok:
```
calls { service.name="shipping", span.name="get_shipping/{shippingId}", span.kind="SERVER", status.code="Ok" }
```
Error counts are computed from the number of spans with an Error status code.

Errors are tracked using a calls metric with a status.code datapoint attribute set to Error:
```
calls { service.name="shipping", span.name="get_shipping/{shippingId}, span.kind="SERVER", status.code="Error" }
```
Duration is computed from the difference between the span start and end times and inserted into the relevant duration histogram time bucket for each unique set dimensions.

Span durations are tracked using a duration histogram metric:
```
duration { service.name="shipping", span.name="get_shipping/{shippingId}", span.kind="SERVER", status.code="Ok" }
```

NOTE: otelcol.connector.spanmetrics is a wrapper over the upstream OpenTelemetry Collector spanmetrics connector. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.connector.spanmetrics components can be specified by giving them different labels.

Usage

otelcol.connector.spanmetrics "LABEL" {
  histogram {
    ...
  }

  output {
    metrics = [...]
  }
}

Arguments

otelcol.connector.spanmetrics supports the following arguments:

Name	Type	Description	Default	Required
`aggregation_temporality`	`string`	Configures whether to reset the metrics after flushing.	`"CUMULATIVE"`	no
`dimensions_cache_size`	`number`	How many dimensions to cache.	`1000`	no
`exclude_dimensions`	`list(string)`	List of dimensions to be excluded from the default set of dimensions.	`[]`	no
`metrics_flush_interval`	`duration`	How often to flush generated metrics.	`"15s"`	no
`namespace`	`string`	Metric namespace.	`""`	no
`resource_metrics_cache_size`	`number`	The size of the cache holding metrics for a service.	`1000`	no
`resource_metrics_key_attributes`	`list(string)`	Limits the resource attributes used to create the metrics.	`[]`	no

Adjusting dimensions_cache_size can improve the Agent process’ memory usage.

The supported values for aggregation_temporality are:

"CUMULATIVE": The metrics will not be reset after they are flushed.
"DELTA": The metrics will be reset after they are flushed.

If namespace is set, the generated metric name will be added a namespace. prefix.

resource_metrics_cache_size is mostly relevant for cumulative temporality. It helps avoid issues with increasing memory and with incorrect metric timestamp resets.

resource_metrics_key_attributes can be used to avoid situations where resource attributes may change across service restarts, causing metric counters to break (and duplicate). A resource does not need to have all of the attributes. The list must include enough attributes to properly identify unique resources or risk aggregating data from more than one service and span. For example, ["service.name", "telemetry.sdk.language", "telemetry.sdk.name"].

Blocks

The following blocks are supported inside the definition of otelcol.connector.spanmetrics:

Hierarchy	Block	Description	Required
dimension	dimension	Dimensions to be added in addition to the default ones.	no
events	events	Configures the events metric.	no
events > dimension	dimension	Span event attributes to add as dimensions to the events metric, on top of the default ones and the ones configured in the top-level `dimension` block.	no
exemplars	exemplars	Configures how to attach exemplars to histograms.	no
histogram	histogram	Configures the histogram derived from spans durations.	yes
histogram > explicit	explicit	Configuration for a histogram with explicit buckets.	no
histogram > exponential	exponential	Configuration for a histogram with exponential buckets.	no
output	output	Configures where to send telemetry data.	yes

It is necessary to specify either a “exponential” or an “explicit” block:

Specifying both an “exponential” and an “explicit” block is not allowed.
Specifying neither an “exponential” nor an “explicit” block is not allowed.

dimension block

The dimension block configures dimensions to be added in addition to the default ones.

The default dimensions are:

service.name
span.name
span.kind
status.code

The default dimensions are always added. If no additional dimensions are specified, only the default ones will be added.

The following attributes are supported:

Name	Type	Description	Default	Required
`default`	`string`	Value to use if the attribute is missing.	null	no
`name`	`string`	Span attribute or resource attribute to look up.		yes

otelcol.connector.spanmetrics will look for the name attribute in the span’s collection of attributes. If it is not found, the resource attributes will be checked.

If the attribute is missing in both the span and resource attributes:

If default is not set, the dimension will be omitted.
If default is set, the dimension will be added and its value will be set to the value of default.

events block

The events block configures the events metric, which tracks span events.

The following attributes are supported:

Name	Type	Description	Default	Required
`enabled`	`bool`	Enables all events metric.	`false`	no

At least one dimension block is required if enabled is set to true.

histogram block

The histogram block configures the histogram derived from spans’ durations.

The following attributes are supported:

Name	Type	Description	Default	Required
`disable`	`bool`	Disable all histogram metrics.	`false`	no
`unit`	`string`	Configures the histogram units.	`"ms"`	no

The supported values for unit are:

"ms": milliseconds
"s": seconds

exponential block

The exponential block configures a histogram with exponential buckets.

The following attributes are supported:

Name	Type	Description	Default	Required
`max_size`	`number`	Maximum number of buckets per positive or negative number range.	`160`	no

explicit block

The explicit block configures a histogram with explicit buckets.

The following attributes are supported:

Name	Type	Description	Default	Required
`buckets`	`list(duration)`	List of histogram buckets.	`["2ms", "4ms", "6ms", "8ms", "10ms", "50ms", "100ms", "200ms", "400ms", "800ms", "1s", "1400ms", "2s", "5s", "10s", "15s"]`	no

exemplars block

The exemplars block configures how to attach exemplars to histograms.

The following attributes are supported:

Name	Type	Description	Default	Required
`enabled`	`bool`	Configures whether to add exemplars to histograms.	`false`	no
`max_per_data_point`	`number`	Limits the number of exemplars that can be added to a unique dimension set.	`null`	no

max_per_data_point can help with reducing memory consumption.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer traces telemetry data. It does not accept metrics and logs.

Handling of resource attributes

otelcol.connector.spanmetrics is an OTLP-native component. As such, it aims to preserve the resource attributes of spans.

For example, let’s assume that there are two incoming resources spans with the same service.name and k8s.pod.name resource attributes.

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "first" }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "trace_id": "7bba9f33312b3dbb8b2c2c62bb7abe2d",
              "span_id": "086e83747d0e381e",
              "name": "TestSpan",
              "attributes": [
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "first" }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "trace_id": "7bba9f33312b3dbb8b2c2c62bb7abe2d",
              "span_id": "086e83747d0e381b",
              "name": "TestSpan",
              "attributes": [
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

otelcol.connector.spanmetrics will preserve the incoming service.name and k8s.pod.name resource attributes by attaching them to the output metrics resource. Only one metric resource will be created, because both span resources have identical resource attributes.

{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "first" }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": { "name": "spanmetricsconnector" },
          "metrics": [
            {
              "name": "calls",
              "sum": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "service.name",
                        "value": { "stringValue": "TestSvcName" }
                      },
                      {
                        "key": "span.name",
                        "value": { "stringValue": "TestSpan" }
                      },
                      {
                        "key": "span.kind",
                        "value": { "stringValue": "SPAN_KIND_UNSPECIFIED" }
                      },
                      {
                        "key": "status.code",
                        "value": { "stringValue": "STATUS_CODE_UNSET" }
                      }
                    ],
                    "startTimeUnixNano": "1702582936761872000",
                    "timeUnixNano": "1702582936761872012",
                    "asInt": "2"
                  }
                ],
                "aggregationTemporality": 2,
                "isMonotonic": true
              }
            }
          ]
        }
      ]
    }
  ]
}

Now assume that otelcol.connector.spanmetrics receives two incoming resource spans, each with a different value for the k8s.pod.name recourse attribute.

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "first" }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "trace_id": "7bba9f33312b3dbb8b2c2c62bb7abe2d",
              "span_id": "086e83747d0e381e",
              "name": "TestSpan",
              "attributes": [
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "second" }
          }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "trace_id": "7bba9f33312b3dbb8b2c2c62bb7abe2d",
              "span_id": "086e83747d0e381b",
              "name": "TestSpan",
              "attributes": [
                {
                  "key": "attribute1",
                  "value": { "intValue": "78" }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

To preserve the values of all resource attributes, otelcol.connector.spanmetrics will produce two resource metrics. Each resource metric will have a different value for the k8s.pod.name recourse attribute. This way none of the resource attributes will be lost during the generation of metrics.

{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "first" }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {
            "name": "spanmetricsconnector"
          },
          "metrics": [
            {
              "name": "calls",
              "sum": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "service.name",
                        "value": { "stringValue": "TestSvcName" }
                      },
                      {
                        "key": "span.name",
                        "value": { "stringValue": "TestSpan" }
                      },
                      {
                        "key": "span.kind",
                        "value": { "stringValue": "SPAN_KIND_UNSPECIFIED" }
                      },
                      {
                        "key": "status.code",
                        "value": { "stringValue": "STATUS_CODE_UNSET" }
                      }
                    ],
                    "startTimeUnixNano": "1702582936761872000",
                    "timeUnixNano": "1702582936761872012",
                    "asInt": "1"
                  }
                ],
                "aggregationTemporality": 2,
                "isMonotonic": true
              }
            }
          ]
        }
      ]
    },
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": { "stringValue": "TestSvcName" }
          },
          {
            "key": "k8s.pod.name",
            "value": { "stringValue": "second" }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {
            "name": "spanmetricsconnector"
          },
          "metrics": [
            {
              "name": "calls",
              "sum": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "service.name",
                        "value": { "stringValue": "TestSvcName" }
                      },
                      {
                        "key": "span.name",
                        "value": { "stringValue": "TestSpan" }
                      },
                      {
                        "key": "span.kind",
                        "value": { "stringValue": "SPAN_KIND_UNSPECIFIED" }
                      },
                      {
                        "key": "status.code",
                        "value": { "stringValue": "STATUS_CODE_UNSET" }
                      }
                    ],
                    "startTimeUnixNano": "1702582936761872000",
                    "timeUnixNano": "1702582936761872012",
                    "asInt": "1"
                  }
                ],
                "aggregationTemporality": 2,
                "isMonotonic": true
              }
            }
          ]
        }
      ]
    }
  ]
}

Component health

otelcol.connector.spanmetrics is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.connector.spanmetrics does not expose any component-specific debug information.

Examples

Explicit histogram and extra dimensions

In the example below, http.status_code and http.method are additional dimensions on top of:

service.name
span.name
span.kind
status.code

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    traces  = [otelcol.connector.spanmetrics.default.input]
  }
}

otelcol.connector.spanmetrics "default" {
  // Since a default is not provided, the http.status_code dimension will be omitted
  // if the span does not contain http.status_code.
  dimension {
    name = "http.status_code"
  }

  // If the span is missing http.method, the connector will insert
  // the http.method dimension with value 'GET'.
  dimension {
    name = "http.method"
    default = "GET"
  }

  dimensions_cache_size = 333

  aggregation_temporality = "DELTA"

  histogram {
    unit = "s"
    explicit {
      buckets = ["333ms", "777s", "999h"]
    }
  }

  // The period on which all metrics (whose dimension keys remain in cache) will be emitted.
  metrics_flush_interval = "33s"

  namespace = "test.namespace"

  output {
    metrics = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
  }
}

Sending metrics via a Prometheus remote write

The generated metrics can be sent to a Prometheus-compatible database such as Grafana Mimir. However, extra steps are required in order to make sure all metric samples are received. This is because otelcol.connector.spanmetrics aims to preserve resource attributes in the metrics which it outputs.

Unfortunately, the Prometheus data model has no notion of resource attributes. This means that if otelcol.connector.spanmetrics outputs metrics with identical metric attributes, but different resource attributes, otelcol.exporter.prometheus will convert the metrics into the same metric series. This problem can be solved by doing either of the following:

Recommended approach: Prior to otelcol.connector.spanmetrics, remove all resource attributes from the incoming spans which are not needed by otelcol.connector.spanmetrics.

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    traces  = [otelcol.processor.transform.default.input]
  }
}

// Remove all resource attributes except the ones which
// the otelcol.connector.spanmetrics needs.
// If this is not done, otelcol.exporter.prometheus may fail to
// write some samples due to an "err-mimir-sample-duplicate-timestamp" error.
// This is because the spanmetricsconnector will create a new
// metrics resource scope for each traces resource scope.
otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "resource"
    statements = [
      // We keep only the "service.name" and "special.attr" resource attributes,
      // because they are the only ones which otelcol.connector.spanmetrics needs.
      //
      // There is no need to list "span.name", "span.kind", and "status.code"
      // here because they are properties of the span (and not resource attributes):
      // https://github.com/open-telemetry/opentelemetry-proto/blob/v1.0.0/opentelemetry/proto/trace/v1/trace.proto
      `keep_keys(attributes, ["service.name", "special.attr"])`,
    ]
  }

  output {
    traces  = [otelcol.connector.spanmetrics.default.input]
  }
}

otelcol.connector.spanmetrics "default" {
  histogram {
    explicit {}
  }

  dimension {
    name = "special.attr"
  }
  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Or, after otelcol.connector.spanmetrics, copy each of the resource attributes as a metric datapoint attribute. This has the advantage that the resource attributes will be visible as metric labels. However, the cardinality of the metrics may be much higher, which could increase the cost of storing and querying them. The example below uses the merge_maps OTTL function.

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    traces  = [otelcol.connector.spanmetrics.default.input]
  }
}

otelcol.connector.spanmetrics "default" {
  histogram {
    explicit {}
  }

  dimension {
    name = "special.attr"
  }
  output {
    metrics = [otelcol.processor.transform.default.input]
  }
}

// Insert resource attributes as metric data point attributes.
otelcol.processor.transform "default" {
  error_mode = "ignore"

  metric_statements {
    context = "datapoint"
    statements = [
      // "insert" means that a metric datapoint attribute will be inserted
      // only if an attribute with the same key does not already exist.
      `merge_maps(attributes, resource.attributes, "insert")`,
    ]
  }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

If the resource attributes are not treated in either of the ways described above, an error such as this one could be logged by prometheus.remote_write: the sample has been rejected because another sample with the same timestamp, but a different value, has already been ingested (err-mimir-sample-duplicate-timestamp).

Note
In order for a Prometheus target_info metric to be generated, the incoming spans resource scope attributes must contain service.name and service.instance.id attributes.

The target_info metric will be generated for each resource scope, while OpenTelemetry metric names and attributes will be normalized to be compliant with Prometheus naming rules.

Compatible components

otelcol.connector.spanmetrics can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.connector.spanmetrics has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.debug

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.debug

otelcol.exporter.debug accepts telemetry data from other otelcol components and writes them to the console (stderr). You can control the verbosity of the logs.

Note
otelcol.exporter.debug is a wrapper over the upstream OpenTelemetry Collector debug exporter. If necessary, bug reports or feature requests are redirected to the upstream repository.

Multiple otelcol.exporter.debug components can be specified by giving them different labels.

Usage

otelcol.exporter.debug "LABEL" { }

Arguments

otelcol.exporter.debug supports the following arguments:

Name	Type	Description	Default	Required
`verbosity`	`string`	Verbosity of the generated logs.	`"normal"`	no
`sampling_initial`	`int`	Number of messages initially logged each second.	`2`	no
`sampling_thereafter`	`int`	Sampling rate after the initial messages are logged.	`500`	no

The verbosity argument must be one of "basic", "normal", or "detailed".

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.exporter.debug is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.debug does not expose any component-specific debug information.

Example

This example scrapes Prometheus UNIX metrics and writes them to the console:

prometheus.exporter.unix "default" { }

prometheus.scrape "default" {
    targets    = prometheus.exporter.unix.default.targets
    forward_to = [otelcol.receiver.prometheus.default.receiver]
}

otelcol.receiver.prometheus "default" {
    output {
        metrics = [otelcol.exporter.debug.default.input]
    }
}

otelcol.exporter.debug "default" {
    verbosity           = "detailed"
    sampling_initial    = 1
    sampling_thereafter = 1
}

Compatible components

otelcol.exporter.debug has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.loadbalancing

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.loadbalancing

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

otelcol.exporter.loadbalancing accepts logs and traces from other otelcol components and writes them over the network using the OpenTelemetry Protocol (OTLP) protocol.

NOTE: otelcol.exporter.loadbalancing is a wrapper over the upstream OpenTelemetry Collector loadbalancing exporter. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.exporter.loadbalancing components can be specified by giving them different labels.

The decision which backend to use depends on the trace ID or the service name. The backend load doesn’t influence the choice. Even though this load-balancer won’t do round-robin balancing of the batches, the load distribution should be very similar among backends, with a standard deviation under 5% at the current configuration.

otelcol.exporter.loadbalancing is especially useful for backends configured with tail-based samplers which choose a backend based on the view of the full trace.

When a list of backends is updated, some of the signals will be rerouted to different backends. Around R/N of the “routes” will be rerouted differently, where:

A “route” is either a trace ID or a service name mapped to a certain backend.
“R” is the total number of routes.
“N” is the total number of backends.

This should be stable enough for most cases, and the larger the number of backends, the less disruption it should cause.

Usage

otelcol.exporter.loadbalancing "LABEL" {
  resolver {
    ...
  }
  protocol {
    otlp {
      client {}
    }
  }
}

Arguments

otelcol.exporter.loadbalancing supports the following arguments:

Name	Type	Description	Default	Required
`routing_key`	`string`	Routing strategy for load balancing.	`"traceID"`	no

The routing_key attribute determines how to route signals across endpoints. Its value could be one of the following:

"service": spans with the same service.name will be exported to the same backend. This is useful when using processors like the span metrics, so all spans for each service are sent to consistent Agent instances for metric collection. Otherwise, metrics for the same services would be sent to different Agents, making aggregations inaccurate.
"traceID": spans belonging to the same traceID will be exported to the same backend.

Blocks

The following blocks are supported inside the definition of otelcol.exporter.loadbalancing:

Hierarchy	Block	Description	Required
resolver	resolver	Configures discovering the endpoints to export to.	yes
resolver > static	static	Static list of endpoints to export to.	no
resolver > dns	dns	DNS-sourced list of endpoints to export to.	no
resolver > kubernetes	kubernetes	Kubernetes-sourced list of endpoints to export to.	no
protocol	protocol	Protocol settings. Only OTLP is supported at the moment.	no
protocol > otlp	otlp	Configures an OTLP exporter.	no
protocol > otlp > client	client	Configures the exporter gRPC client.	no
protocol > otlp > client > tls	tls	Configures TLS for the gRPC client.	no
protocol > otlp > client > keepalive	keepalive	Configures keepalive settings for the gRPC client.	no
protocol > otlp > queue	queue	Configures batching of data before sending.	no
protocol > otlp > retry	retry	Configures retry mechanism for failed requests.	no
debug_metrics	debug_metrics	Configures the metrics that this component generates to monitor its state.	no

The > symbol indicates deeper levels of nesting. For example, resolver > static refers to a static block defined inside a resolver block.

resolver block

The resolver block configures how to retrieve the endpoint to which this exporter will send data.

Inside the resolver block, either the dns block or the static block should be specified. If both dns and static are specified, dns takes precedence.

static block

The static block configures a list of endpoints which this exporter will send data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`hostnames`	`list(string)`	List of endpoints to export to.		yes

dns block

The dns block periodically resolves an IP address via the DNS hostname attribute. This IP address and the port specified via the port attribute will then be used by the gRPC exporter as the endpoint to which to export data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`hostname`	`string`	DNS hostname to resolve.		yes
`interval`	`duration`	Resolver interval.	`"5s"`	no
`timeout`	`duration`	Resolver timeout.	`"1s"`	no
`port`	`string`	Port to be used with the IP addresses resolved from the DNS hostname.	`"4317"`	no

kubernetes block

You can use the kubernetes block to load balance across the pods of a Kubernetes service. The Kubernetes API notifies Grafana Agent Flow whenever a new pod is added or removed from the service. The kubernetes resolver has a much faster response time than the dns resolver because it doesn’t require polling.

The following arguments are supported:

Name	Type	Description	Default	Required
`service`	`string`	Kubernetes service to resolve.		yes
`ports`	`list(number)`	Ports to use with the IP addresses resolved from `service`.	`[4317]`	no

If no namespace is specified inside service, an attempt will be made to infer the namespace for this Agent. If this fails, the default namespace will be used.

Each of the ports listed in ports will be used with each of the IPs resolved from service.

The “get”, “list”, and “watch” roles must be granted in Kubernetes for the resolver to work.

protocol block

The protocol block configures protocol-related settings for exporting. At the moment only the OTLP protocol is supported.

otlp block

The otlp block configures OTLP-related settings for exporting.

client block

The client block configures the gRPC client used by the component. The endpoints used by the client block are the ones from the resolver block

The following arguments are supported:

Name	Type	Description	Default	Required
`compression`	`string`	Compression mechanism to use for requests.	`"gzip"`	no
`read_buffer_size`	`string`	Size of the read buffer the gRPC client to use for reading server responses.		no
`write_buffer_size`	`string`	Size of the write buffer the gRPC client to use for writing requests.	`"512KiB"`	no
`wait_for_ready`	`boolean`	Waits for gRPC connection to be in the `READY` state before sending data.	`false`	no
`headers`	`map(string)`	Additional headers to send with the request.	`{}`	no
`balancer_name`	`string`	Which gRPC client-side load balancer to use for requests.	`pick_first`	no
`authority`	`string`	Overrides the default `:authority` header in gRPC requests from the gRPC client.		no
`auth`	`capsule(otelcol.Handler)`	Handler from an `otelcol.auth` component to use for authenticating requests.		no

By default, requests are compressed with gzip. The compression argument controls which compression mechanism to use. Supported strings are:

"gzip"
"zlib"
"deflate"
"snappy"
"zstd"

If compression is set to "none" or an empty string "", no compression is used.

The supported values for balancer_name are listed in the gRPC documentation on Load balancing:

pick_first: Tries to connect to the first address, uses it for all RPCs if it connects, or tries the next address if it fails (and keeps doing that until one connection is successful). Because of this, all the RPCs will be sent to the same backend.
round_robin: Connects to all the addresses it sees and sends an RPC to each backend one at a time in order. For example, the first RPC is sent to backend-1, the second RPC is sent to backend-2, and the third RPC is sent to backend-1.

The :authority header in gRPC specifies the host to which the request is being sent. It’s similar to the Host header in HTTP requests. By default, the value for :authority is derived from the endpoint URL used for the gRPC call. Overriding :authority could be useful when routing traffic using a proxy like Envoy, which makes routing decisions based on the value of the :authority header.

You can configure an HTTP proxy with the following environment variables:

HTTPS_PROXY
NO_PROXY

The HTTPS_PROXY environment variable specifies a URL to use for proxying requests. Connections to the proxy are established via the HTTP CONNECT method.

The NO_PROXY environment variable is an optional list of comma-separated hostnames for which the HTTPS proxy should not be used. Each hostname can be provided as an IP address (1.2.3.4), an IP address in CIDR notation (1.2.3.4/8), a domain name (example.com), or *. A domain name matches that domain and all subdomains. A domain name with a leading “.” (.example.com) matches subdomains only. NO_PROXY is only read when HTTPS_PROXY is set.

Because otelcol.exporter.loadbalancing uses gRPC, the configured proxy server must be able to handle and proxy HTTP/2 traffic.

tls block

The tls block configures TLS settings used for the connection to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

keepalive block

The keepalive block configures keepalive settings for gRPC client connections.

The following arguments are supported:

Name	Type	Description	Required
`ping_wait`	`duration`	How often to ping the server after no activity.	no
`ping_response_timeout`	`duration`	Time to wait before closing inactive connections if the server does not respond to a ping.	no
`ping_without_stream`	`boolean`	Send pings even if there is no active stream request.	no

queue block

The queue block configures an in-memory buffer of batches before data is sent to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables an in-memory buffer before sending data to the client.	`true`	no
`num_consumers`	`number`	Number of readers to send batches written to the queue in parallel.	`10`	no
`queue_size`	`number`	Maximum number of unwritten batches allowed in the queue at the same time.	`1000`	no

When enabled is true, data is first written to an in-memory buffer before sending it to the configured server. Batches sent to the component’s input exported field are added to the buffer as long as the number of unsent batches doesn’t exceed the configured queue_size.

queue_size determines how long an endpoint outage is tolerated. Assuming 100 requests/second, the default queue size 1000 provides about 10 seconds of outage tolerance. To calculate the correct value for queue_size, multiply the average number of outgoing requests per second by the time in seconds that outages are tolerated. A very high value can cause Out Of Memory (OOM) kills.

The num_consumers argument controls how many readers read from the buffer and send data in parallel. Larger values of num_consumers allow data to be sent more quickly at the expense of increased network traffic.

retry block

The retry block configures how failed requests to the gRPC server are retried.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables retrying failed requests.	`true`	no
`initial_interval`	`duration`	Initial time to wait before retrying a failed request.	`"5s"`	no
`max_elapsed_time`	`duration`	Maximum time to wait before discarding a failed batch.	`"5m"`	no
`max_interval`	`duration`	Maximum time to wait between retries.	`"30s"`	no
`multiplier`	`number`	Factor to grow wait time before retrying.	`1.5`	no
`randomization_factor`	`number`	Factor to randomize wait time before retrying.	`0.5`	no

When enabled is true, failed batches are retried after a given interval. The initial_interval argument specifies how long to wait before the first retry attempt. If requests continue to fail, the time to wait before retrying increases by the factor specified by the multiplier argument, which must be greater than 1.0. The max_interval argument specifies the upper bound of how long to wait between retries.

The randomization_factor argument is useful for adding jitter between retrying agents. If randomization_factor is greater than 0, the wait time before retries is multiplied by a random factor in the range [ I - randomization_factor * I, I + randomization_factor * I], where I is the current interval.

If a batch hasn’t been sent successfully, it is discarded after the time specified by max_elapsed_time elapses. If max_elapsed_time is set to "0s", failed requests are retried forever until they succeed.

debug_metrics block

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

Name	Type	Description	Default	Required
`disable_high_cardinality_metrics`	`boolean`	Whether to disable certain high cardinality metrics.	`true`	no

disable_high_cardinality_metrics is the Grafana Agent equivalent to the telemetry.disableHighCardinalityMetrics feature gate in the OpenTelemetry Collector. It removes attributes that could cause high cardinality metrics. For example, attributes with IP addresses and port numbers in metrics about HTTP and gRPC connections are removed.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer OTLP-formatted data for telemetry signals of these types:

logs
traces

Choose a load balancing strategy

Different Grafana Agent Flow components require different load-balancing strategies. The use of otelcol.exporter.loadbalancing is only necessary for stateful Flow components.

otelcol.processor.tail_sampling

All spans for a given trace ID must go to the same tail sampling Grafana Agent instance.

This can be done by configuring otelcol.exporter.loadbalancing with routing_key = "traceID".
If you do not configure routing_key = "traceID", the sampling decision may be incorrect. The tail sampler must have a full view of the trace when making a sampling decision. For example, a rate_limiting tail sampling strategy may incorrectly pass through more spans than expected if the spans for the same trace are spread out to more than one Grafana Agent Flow instance.

otelcol.connector.spanmetrics

All spans for a given service.name must go to the same spanmetrics Grafana Agent.

This can be done by configuring otelcol.exporter.loadbalancing with routing_key = "service".
If you do not configure routing_key = "service", metrics generated from spans might be incorrect. For example, if similar spans for the same service.name end up on different Grafana Agent instances, the two Grafana Agents will have identical metric series for calculating span latency, errors, and number of requests. When both Grafana Agent instances attempt to write the metrics to a database such as Mimir, the series may clash with each other. At best, this will lead to an error in Grafana Agent and a rejected write to the metrics database. At worst, it could lead to inaccurate data due to overlapping samples for the metric series.

However, there are ways to scale otelcol.connector.spanmetrics without the need for a load balancer:

Each Grafana Agent could add an attribute such as collector.id in order to make its series unique. Then, for example, you could use a sum by PromQL query to aggregate the metrics from different Grafana Agents. Unfortunately, an extra collector.id attribute has a downside that the metrics stored in the database will have higher cardinality.
Spanmetrics could be generated in the backend database instead of in Grafana Agent. For example, span metrics can be generated in Grafana Cloud by the Tempo traces database.

otelcol.connector.servicegraph

It is challenging to scale otelcol.connector.servicegraph over multiple Grafana Agent instances. For otelcol.connector.servicegraph to work correctly, each “client” span must be paired with a “server” span to calculate metrics such as span duration. If a “client” span goes to one Grafana Agent, but a “server” span goes to another Grafana Agent, then no single Grafana Agent will be able to pair the spans and a metric won’t be generated.

otelcol.exporter.loadbalancing can solve this problem partially if it is configured with routing_key = "traceID". Each Grafana Agent will then be able to calculate a service graph for each “client”/“server” pair in a trace. It is possible to have a span with similar “server”/“client” values in a different trace, processed by another Grafana Agent. If two different Grafana Agent instances process similar “server”/“client” spans, they will generate the same service graph metric series. If the series from two Grafana Agent are the same, this will lead to issues when writing them to the backend database. You could differentiate the series by adding an attribute such as "collector.id". The series from different Grafana Agents can be aggregated using PromQL queries on the backed metrics database. If the metrics are stored in Grafana Mimir, cardinality issues due to "collector.id" labels can be solved using Adaptive Metrics.

A simpler, more scalable alternative to generating service graph metrics in Grafana Agent is to generate them entirely in the backend database. For example, service graphs can be generated in Grafana Cloud by the Tempo traces database.

Mixing stateful components

Different Grafana Agent Flow components may require a different routing_key for otelcol.exporter.loadbalancing. For example, otelcol.processor.tail_sampling requires routing_key = "traceID" whereas otelcol.connector.spanmetrics requires routing_key = "service". To load balance both types of components, two different sets of load balancers have to be set up:

One set of otelcol.exporter.loadbalancing with routing_key = "traceID", sending spans to Grafana Agents doing tail sampling and no span metrics.
Another set of otelcol.exporter.loadbalancing with routing_key = "service", sending spans to Grafana Agents doing span metrics and no service graphs.

Unfortunately, this can also lead to side effects. For example, if otelcol.connector.spanmetrics is configured to generate exemplars, the tail sampling Grafana Agents might drop the trace that the exemplar points to. There is no coordination between the tail sampling Grafana Agents and the span metrics Grafana Agents to make sure trace IDs for exemplars are kept.

Component health

otelcol.exporter.loadbalancing is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.loadbalancing does not expose any component-specific debug information.

Examples

Static resolver

This example accepts OTLP logs and traces over gRPC. It then sends them in a load-balanced way to “localhost:55690” or “localhost:55700”.

otelcol.receiver.otlp "default" {
    grpc {}
    output {
        traces  = [otelcol.exporter.loadbalancing.default.input]
        logs    = [otelcol.exporter.loadbalancing.default.input]
    }
}

otelcol.exporter.loadbalancing "default" {
    resolver {
        static {
            hostnames = ["localhost:55690", "localhost:55700"]
        }
    }
    protocol {
        otlp {
            client {}
        }
    }
}

DNS resolver

When configured with a dns resolver, otelcol.exporter.loadbalancing will do a DNS lookup on regular intervals. Spans are exported to the addresses the DNS lookup returned.

otelcol.exporter.loadbalancing "default" {
    resolver {
        dns {
            hostname = "grafana-agent-traces-sampling.grafana-cloud-monitoring.svc.cluster.local"
            port     = "34621"
            interval = "5s"
            timeout  = "1s"
        }
    }
    protocol {
        otlp {
            client {}
        }
    }
}

The following example shows a Kubernetes configuration that configures two sets of Grafana Agents:

A pool of load-balancer Grafana Agents:
- Spans are received from instrumented applications via otelcol.receiver.otlp
- Spans are exported via otelcol.exporter.loadbalancing.
A pool of sampling Grafana Agents:
- The sampling Grafana Agents run behind a headless service to enable the load-balancer Grafana Agents to discover them.
- Spans are received from the load-balancer Grafana Agents via otelcol.receiver.otlp
- Traces are sampled via otelcol.processor.tail_sampling.
- The traces are exported via otelcol.exporter.otlp to an OTLP-compatible database such as Tempo.

apiVersion: v1
kind: Namespace
metadata:
  name: grafana-cloud-monitoring
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: k6-trace-generator
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: k6-trace-generator
  template:
    metadata:
      labels:
        name: k6-trace-generator
    spec:
      containers:
      - env:
        - name: ENDPOINT
          value: agent-traces-lb.grafana-cloud-monitoring.svc.cluster.local:9411
        image: ghcr.io/grafana/xk6-client-tracing:v0.0.2
        imagePullPolicy: IfNotPresent
        name: k6-trace-generator
---
apiVersion: v1
kind: Service
metadata:
  name: agent-traces-lb
  namespace: grafana-cloud-monitoring
spec:
  clusterIP: None
  ports:
  - name: agent-traces-otlp-grpc
    port: 9411
    protocol: TCP
    targetPort: 9411
  selector:
    name: agent-traces-lb
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-traces-lb
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: agent-traces-lb
  template:
    metadata:
      labels:
        name: agent-traces-lb
    spec:
      containers:
      - args:
        - run
        - /etc/agent/agent_lb.river
        command:
        - /bin/grafana-agent
        env:
        - name: AGENT_MODE
          value: flow
        image: grafana/agent:v0.38.0
        imagePullPolicy: IfNotPresent
        name: agent-traces
        ports:
        - containerPort: 9411
          name: otlp-grpc
          protocol: TCP
        - containerPort: 34621
          name: agent-lb
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/agent
          name: agent-traces
      volumes:
      - configMap:
          name: agent-traces
        name: agent-traces
---
apiVersion: v1
kind: Service
metadata:
  name: agent-traces-sampling
  namespace: grafana-cloud-monitoring
spec:
  clusterIP: None
  ports:
  - name: agent-lb
    port: 34621
    protocol: TCP
    targetPort: agent-lb
  selector:
    name: agent-traces-sampling
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-traces-sampling
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 3
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: agent-traces-sampling
  template:
    metadata:
      labels:
        name: agent-traces-sampling
    spec:
      containers:
      - args:
        - run
        - /etc/agent/agent_sampling.river
        command:
        - /bin/grafana-agent
        env:
        - name: AGENT_MODE
          value: flow
        image: grafana/agent:v0.38.0
        imagePullPolicy: IfNotPresent
        name: agent-traces
        ports:
        - containerPort: 9411
          name: otlp-grpc
          protocol: TCP
        - containerPort: 34621
          name: agent-lb
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/agent
          name: agent-traces
      volumes:
      - configMap:
          name: agent-traces
        name: agent-traces
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: agent-traces
  namespace: grafana-cloud-monitoring
data:
  agent_lb.river: |
    otelcol.receiver.otlp "default" {
      grpc {
        endpoint = "0.0.0.0:9411"
      }
      output {
        traces = [otelcol.exporter.loadbalancing.default.input,otelcol.exporter.logging.default.input]
      }
    }

    otelcol.exporter.logging "default" {
      verbosity = "detailed"
    }

    otelcol.exporter.loadbalancing "default" {
      resolver {
        dns {
          hostname = "agent-traces-sampling.grafana-cloud-monitoring.svc.cluster.local"
          port = "34621"
        }
      }
      protocol {
        otlp {
          client {
            tls {
              insecure = true
            }
          }
        }
      }
    }

  agent_sampling.river: |
    otelcol.receiver.otlp "default" {
      grpc {
        endpoint = "0.0.0.0:34621"
      }
      output {
        traces = [otelcol.exporter.otlp.default.input,otelcol.exporter.logging.default.input]
      }
    }

    otelcol.exporter.logging "default" {
      verbosity = "detailed"
    }

    otelcol.exporter.otlp "default" {
      client {
        endpoint = "tempo-prod-06-prod-gb-south-0.grafana.net:443"
        auth     = otelcol.auth.basic.creds.handler
      }
    }

    otelcol.auth.basic "creds" {
      username = "111111"
      password = "pass"
    }

You must fill in the correct OTLP credentials prior to running the example. You can use k3d to start the example:

k3d cluster create grafana-agent-lb-test
kubectl apply -f kubernetes_config.yaml

To delete the cluster, run:

k3d cluster delete grafana-agent-lb-test

Kubernetes resolver

When you configure otelcol.exporter.loadbalancing with a kubernetes resolver, the Kubernetes API notifies Grafana Agent Flow whenever a new pod is added or removed from the service. Spans are exported to the addresses from the Kubernetes API, combined with all the possible ports.

otelcol.exporter.loadbalancing "default" {
    resolver {
        kubernetes {
            service = "grafana-agent-traces-headless"
            ports   = [ 34621 ]
        }
    }
    protocol {
        otlp {
            client {}
        }
    }
}

The following example shows a Kubernetes configuration that sets up two sets of Grafana Agents:

A pool of load-balancer Grafana Agents:
- Spans are received from instrumented applications via otelcol.receiver.otlp
- Spans are exported via otelcol.exporter.loadbalancing.
- The load-balancer Grafana Agents will get notified by the Kubernetes API any time a pod is added or removed from the pool of sampling Grafana Agents.
A pool of sampling Grafana Agents:
- The sampling Grafana Agents do not need to run behind a headless service.
- Spans are received from the load-balancer Grafana Agents via otelcol.receiver.otlp
- Traces are sampled via otelcol.processor.tail_sampling.
- The traces are exported via otelcol.exporter.otlp to a an OTLP-compatible database such as Tempo.

apiVersion: v1
kind: Namespace
metadata:
  name: grafana-cloud-monitoring
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: agent-traces
  namespace: grafana-cloud-monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: agent-traces-role
  namespace: grafana-cloud-monitoring
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: agent-traces-rolebinding
  namespace: grafana-cloud-monitoring
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: agent-traces-role
subjects:
- kind: ServiceAccount
  name: agent-traces
  namespace: grafana-cloud-monitoring
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: k6-trace-generator
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: k6-trace-generator
  template:
    metadata:
      labels:
        name: k6-trace-generator
    spec:
      containers:
      - env:
        - name: ENDPOINT
          value: agent-traces-lb.grafana-cloud-monitoring.svc.cluster.local:9411
        image: ghcr.io/grafana/xk6-client-tracing:v0.0.2
        imagePullPolicy: IfNotPresent
        name: k6-trace-generator
---
apiVersion: v1
kind: Service
metadata:
  name: agent-traces-lb
  namespace: grafana-cloud-monitoring
spec:
  clusterIP: None
  ports:
  - name: agent-traces-otlp-grpc
    port: 9411
    protocol: TCP
    targetPort: 9411
  selector:
    name: agent-traces-lb
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-traces-lb
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: agent-traces-lb
  template:
    metadata:
      labels:
        name: agent-traces-lb
    spec:
      containers:
      - args:
        - run
        - /etc/agent/agent_lb.river
        command:
        - /bin/grafana-agent
        env:
        - name: AGENT_MODE
          value: flow
        image: grafana/agent:v0.38.0
        imagePullPolicy: IfNotPresent
        name: agent-traces
        ports:
        - containerPort: 9411
          name: otlp-grpc
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/agent
          name: agent-traces
      serviceAccount: agent-traces
      volumes:
      - configMap:
          name: agent-traces
        name: agent-traces
---
apiVersion: v1
kind: Service
metadata:
  name: agent-traces-sampling
  namespace: grafana-cloud-monitoring
spec:
  ports:
  - name: agent-lb
    port: 34621
    protocol: TCP
    targetPort: agent-lb
  selector:
    name: agent-traces-sampling
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-traces-sampling
  namespace: grafana-cloud-monitoring
spec:
  minReadySeconds: 10
  replicas: 3
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      name: agent-traces-sampling
  template:
    metadata:
      labels:
        name: agent-traces-sampling
    spec:
      containers:
      - args:
        - run
        - /etc/agent/agent_sampling.river
        command:
        - /bin/grafana-agent
        env:
        - name: AGENT_MODE
          value: flow
        image: grafana/agent:v0.38.0
        imagePullPolicy: IfNotPresent
        name: agent-traces
        ports:
        - containerPort: 34621
          name: agent-lb
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/agent
          name: agent-traces
      volumes:
      - configMap:
          name: agent-traces
        name: agent-traces
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: agent-traces
  namespace: grafana-cloud-monitoring
data:
  agent_lb.river: |
    otelcol.receiver.otlp "default" {
      grpc {
        endpoint = "0.0.0.0:9411"
      }
      output {
        traces = [otelcol.exporter.loadbalancing.default.input,otelcol.exporter.logging.default.input]
      }
    }

    otelcol.exporter.logging "default" {
      verbosity = "detailed"
    }

    otelcol.exporter.loadbalancing "default" {
      resolver {
        kubernetes {
          service = "agent-traces-sampling"
          ports = ["34621"]
        }
      }
      protocol {
        otlp {
          client {
            tls {
              insecure = true
            }
          }
        }
      }
    }

  agent_sampling.river: |
    otelcol.receiver.otlp "default" {
      grpc {
        endpoint = "0.0.0.0:34621"
      }
      output {
        traces = [otelcol.exporter.otlp.default.input,otelcol.exporter.logging.default.input]
      }
    }

    otelcol.exporter.logging "default" {
      verbosity = "detailed"
    }

    otelcol.exporter.otlp "default" {
      client {
        endpoint = "tempo-prod-06-prod-gb-south-0.grafana.net:443"
        auth     = otelcol.auth.basic.creds.handler
      }
    }

    otelcol.auth.basic "creds" {
      username = "111111"
      password = "pass"
    }

You must fill in the correct OTLP credentials prior to running the example. You can use k3d to start the example:

k3d cluster create grafana-agent-lb-test
kubectl apply -f kubernetes_config.yaml

To delete the cluster, run:

k3d cluster delete grafana-agent-lb-test

Compatible components

otelcol.exporter.loadbalancing has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.logging

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.logging

otelcol.exporter.logging accepts telemetry data from other otelcol components and writes them to the console.

This component writes logs at the info level. The logging config block must be configured to write logs at the info level.

NOTE: otelcol.exporter.logging is a wrapper over the upstream OpenTelemetry Collector logging exporter. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.exporter.logging components can be specified by giving them different labels.

Usage

otelcol.exporter.logging "LABEL" { }

Arguments

otelcol.exporter.logging supports the following arguments:

Name	Type	Description	Default	Required
`verbosity`	`string`	Verbosity of the generated logs.	`"normal"`	no
`sampling_initial`	`int`	Number of messages initially logged each second.	`2`	no
`sampling_thereafter`	`int`	Sampling rate after the initial messages are logged.	`500`	no

The verbosity argument must be one of "basic", "normal", or "detailed".

Blocks

The following blocks are supported inside the definition of otelcol.exporter.logging:

Hierarchy	Block	Description	Required
debug_metrics	debug_metrics	Configures the metrics that this component generates to monitor its state.	no

The > symbol indicates deeper levels of nesting. For example, client > tls refers to a tls block defined inside a client block.

debug_metrics block

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

Name	Type	Description	Default	Required
`disable_high_cardinality_metrics`	`boolean`	Whether to disable certain high cardinality metrics.	`true`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.exporter.logging is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.logging does not expose any component-specific debug information.

Example

This example scrapes prometheus unix metrics and writes them to the console:

prometheus.exporter.unix "default" { }

prometheus.scrape "default" {
    targets    = prometheus.exporter.unix.default.targets
    forward_to = [otelcol.receiver.prometheus.default.receiver]
}

otelcol.receiver.prometheus "default" {
    output {
        metrics = [otelcol.exporter.logging.default.input]
    }
}

otelcol.exporter.logging "default" {
    verbosity           = "detailed"
    sampling_initial    = 1
    sampling_thereafter = 1
}

Compatible components

otelcol.exporter.logging has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.loki

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.loki

otelcol.exporter.loki accepts OTLP-formatted logs from other otelcol components, converts them to Loki-formatted log entries, and forwards them to loki components.

NOTE: otelcol.exporter.loki is a custom component unrelated to the lokiexporter from the OpenTelemetry Collector.

The attributes of the OTLP log are not converted to Loki attributes by default. To convert them, the OTLP log should contain special “hint” attributes:

To convert OTLP resource attributes to Loki labels, use the loki.resource.labels hint attribute.
To convert OTLP log attributes to Loki labels, use the loki.attribute.labels hint attribute.

Labels will be translated to a [Prometheus format][], which is more constrained than the OTLP format. For examples on label translation, see the Converting OTLP attributes to Loki labels section.

Multiple otelcol.exporter.loki components can be specified by giving them different labels.

Usage

otelcol.exporter.loki "LABEL" {
  forward_to = [...]
}

Arguments

otelcol.exporter.loki supports the following arguments:

Name	Type	Description	Default	Required
`forward_to`	`list(receiver)`	Where to forward converted Loki logs.		yes

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for logs. Other telemetry signals are ignored.

Logs sent to input are converted to Loki-compatible log entries and are forwarded to the forward_to argument in sequence.

Component health

otelcol.exporter.loki is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.loki does not expose any component-specific debug information.

Examples

Basic usage

This example accepts OTLP logs over gRPC, transforms them and forwards the converted log entries to loki.write:

otelcol.receiver.otlp "default" {
  grpc {}

  output {
    logs = [otelcol.exporter.loki.default.input]
  }
}

otelcol.exporter.loki "default" {
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100"
  }
}

Converting OTLP attributes to Loki labels

The example below will convert the following attributes to Loki labels:

The service.name and service.namespace OTLP resource attributes.
The event.domain and event.name OTLP log attributes.

Labels will be translated to a [Prometheus format][]. For example:

OpenTelemetry Attribute	Prometheus Label
`name`	`name`
`host.name`	`host_name`
`host_name`	`host_name`
`name (of the host)`	`name__of_the_host_`
`2 cents`	`key_2_cents`
`__name`	`__name`
`_name`	`key_name`
`_name`	`_name` (if `PermissiveLabelSanitization` is enabled)

otelcol.receiver.otlp "default" {
  grpc {}

  output {
    logs = [otelcol.processor.attributes.default.input]
  }
}

otelcol.processor.attributes "default" {
  action {
    key = "loki.attribute.labels"
    action = "insert"
    value = "event.domain, event.name"
  }
  
  action {
    key = "loki.resource.labels"
    action = "insert"
    value = "service.name, service.namespace"
  }
  
  output {
    logs = [otelcol.exporter.loki.default.input]
  }
}

otelcol.exporter.loki "default" {
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
      url = "loki:3100"
  }
}

Prometheus format

Compatible components

otelcol.exporter.loki can accept arguments from the following components:

Components that export Loki LogsReceiver

otelcol.exporter.loki has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.otlp

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.otlp

otelcol.exporter.otlp accepts telemetry data from other otelcol components and writes them over the network using the OTLP gRPC protocol.

NOTE: otelcol.exporter.otlp is a wrapper over the upstream OpenTelemetry Collector otlp exporter. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.exporter.otlp components can be specified by giving them different labels.

Usage

otelcol.exporter.otlp "LABEL" {
  client {
    endpoint = "HOST:PORT"
  }
}

Arguments

otelcol.exporter.otlp supports the following arguments:

Name	Type	Description	Default	Required
`timeout`	`duration`	Time to wait before marking a request as failed.	`"5s"`	no

Blocks

The following blocks are supported inside the definition of otelcol.exporter.otlp:

Hierarchy	Block	Description	Required
client	client	Configures the gRPC server to send telemetry data to.	yes
client > tls	tls	Configures TLS for the gRPC client.	no
client > keepalive	keepalive	Configures keepalive settings for the gRPC client.	no
sending_queue	sending_queue	Configures batching of data before sending.	no
retry_on_failure	retry_on_failure	Configures retry mechanism for failed requests.	no
debug_metrics	debug_metrics	Configures the metrics that this component generates to monitor its state.	no

The > symbol indicates deeper levels of nesting. For example, client > tls refers to a tls block defined inside a client block.

client block

The client block configures the gRPC client used by the component.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to send telemetry data to.		yes
`compression`	`string`	Compression mechanism to use for requests.	`"gzip"`	no
`read_buffer_size`	`string`	Size of the read buffer the gRPC client to use for reading server responses.		no
`write_buffer_size`	`string`	Size of the write buffer the gRPC client to use for writing requests.	`"512KiB"`	no
`wait_for_ready`	`boolean`	Waits for gRPC connection to be in the `READY` state before sending data.	`false`	no
`headers`	`map(string)`	Additional headers to send with the request.	`{}`	no
`balancer_name`	`string`	Which gRPC client-side load balancer to use for requests.	`pick_first`	no
`authority`	`string`	Overrides the default `:authority` header in gRPC requests from the gRPC client.		no
`auth`	`capsule(otelcol.Handler)`	Handler from an `otelcol.auth` component to use for authenticating requests.		no

By default, requests are compressed with gzip. The compression argument controls which compression mechanism to use. Supported strings are:

"gzip"
"zlib"
"deflate"
"snappy"
"zstd"

If compression is set to "none" or an empty string "", no compression is used.

The supported values for balancer_name are listed in the gRPC documentation on Load balancing:

pick_first: Tries to connect to the first address, uses it for all RPCs if it connects, or tries the next address if it fails (and keeps doing that until one connection is successful). Because of this, all the RPCs will be sent to the same backend.
round_robin: Connects to all the addresses it sees and sends an RPC to each backend one at a time in order. For example, the first RPC is sent to backend-1, the second RPC is sent to backend-2, and the third RPC is sent to backend-1.

An HTTP proxy can be configured through the following environment variables:

HTTPS_PROXY
NO_PROXY

The HTTPS_PROXY environment variable specifies a URL to use for proxying requests. Connections to the proxy are established via the HTTP CONNECT method.

Because otelcol.exporter.otlp uses gRPC, the configured proxy server must be able to handle and proxy HTTP/2 traffic.

tls block

The tls block configures TLS settings used for the connection to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

NOTE: otelcol.exporter.otlp uses gRPC, which does not allow you to send sensitive credentials (like auth) over insecure channels. Sending sensitive credentials over insecure non-TLS connections is supported by non-gRPC exporters such as otelcol.exporter.otlphttp.

keepalive block

The keepalive block configures keepalive settings for gRPC client connections.

The following arguments are supported:

Name	Type	Description	Required
`ping_wait`	`duration`	How often to ping the server after no activity.	no
`ping_response_timeout`	`duration`	Time to wait before closing inactive connections if the server does not respond to a ping.	no
`ping_without_stream`	`boolean`	Send pings even if there is no active stream request.	no

sending_queue block

The sending_queue block configures an in-memory buffer of batches before data is sent to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables an in-memory buffer before sending data to the client.	`true`	no
`num_consumers`	`number`	Number of readers to send batches written to the queue in parallel.	`10`	no
`queue_size`	`number`	Maximum number of unwritten batches allowed in the queue at the same time.	`1000`	no

retry_on_failure block

The retry_on_failure block configures how failed requests to the gRPC server are retried.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables retrying failed requests.	`true`	no
`initial_interval`	`duration`	Initial time to wait before retrying a failed request.	`"5s"`	no
`max_elapsed_time`	`duration`	Maximum time to wait before discarding a failed batch.	`"5m"`	no
`max_interval`	`duration`	Maximum time to wait between retries.	`"30s"`	no
`multiplier`	`number`	Factor to grow wait time before retrying.	`1.5`	no
`randomization_factor`	`number`	Factor to randomize wait time before retrying.	`0.5`	no

debug_metrics block

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

Name	Type	Description	Default	Required
`disable_high_cardinality_metrics`	`boolean`	Whether to disable certain high cardinality metrics.	`true`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.exporter.otlp is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.otlp does not expose any component-specific debug information.

Debug metrics

exporter_sent_spans_ratio_total (counter): Number of spans successfully sent to destination.
exporter_send_failed_spans_ratio_total (counter): Number of spans in failed attempts to send to destination.
exporter_queue_capacity_ratio (gauge): Fixed capacity of the retry queue (in batches)
exporter_queue_size_ratio (gauge): Current size of the retry queue (in batches)
rpc_client_duration_milliseconds (histogram): Measures the duration of inbound RPC.
rpc_client_request_size_bytes (histogram): Measures size of RPC request messages (uncompressed).
rpc_client_requests_per_rpc (histogram): Measures the number of messages received per RPC. Should be 1 for all non-streaming RPCs.
rpc_client_response_size_bytes (histogram): Measures size of RPC response messages (uncompressed).
rpc_client_responses_per_rpc (histogram): Measures the number of messages received per RPC. Should be 1 for all non-streaming RPCs.

Examples

The following examples show you how to create an exporter to send data to different destinations.

Send data to a local Tempo instance

You can create an exporter that sends your data to a local Grafana Tempo instance without TLS:

otelcol.exporter.otlp "tempo" {
    client {
        endpoint = "tempo:4317"
        tls {
            insecure             = true
            insecure_skip_verify = true
        }
    }
}

Send data to a managed service

You can create an otlp exporter that sends your data to a managed service, for example, Grafana Cloud. The Tempo username and Grafana Cloud API Key are injected in this example through environment variables.

otelcol.exporter.otlp "grafana_cloud_tempo" {
    client {
        endpoint = "tempo-xxx.grafana.net/tempo:443"
        auth     = otelcol.auth.basic.grafana_cloud_tempo.handler
    }
}
otelcol.auth.basic "grafana_cloud_tempo" {
    username = env("TEMPO_USERNAME")
    password = env("GRAFANA_CLOUD_API_KEY")
}

Compatible components

otelcol.exporter.otlp has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.otlphttp

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.otlphttp

otelcol.exporter.otlphttp accepts telemetry data from other otelcol components and writes them over the network using the OTLP HTTP protocol.

NOTE: otelcol.exporter.otlphttp is a wrapper over the upstream OpenTelemetry Collector otlphttp exporter. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.exporter.otlphttp components can be specified by giving them different labels.

Usage

otelcol.exporter.otlphttp "LABEL" {
  client {
    endpoint = "HOST:PORT"
  }
}

Arguments

otelcol.exporter.otlphttp supports the following arguments:

Name	Type	Description	Default	Required
`metrics_endpoint`	`string`	The endpoint to send metrics to.	`client.endpoint + "/v1/metrics"`	no
`logs_endpoint`	`string`	The endpoint to send logs to.	`client.endpoint + "/v1/logs"`	no
`traces_endpoint`	`string`	The endpoint to send traces to.	`client.endpoint + "/v1/traces"`	no

The default value depends on the endpoint field set in the required client block. If set, these arguments override the client.endpoint field for the corresponding signal.

Blocks

The following blocks are supported inside the definition of otelcol.exporter.otlphttp:

Hierarchy	Block	Description	Required
client	client	Configures the HTTP server to send telemetry data to.	yes
client > tls	tls	Configures TLS for the HTTP client.	no
sending_queue	sending_queue	Configures batching of data before sending.	no
retry_on_failure	retry_on_failure	Configures retry mechanism for failed requests.	no
debug_metrics	debug_metrics	Configures the metrics that this component generates to monitor its state.	no

The > symbol indicates deeper levels of nesting. For example, client > tls refers to a tls block defined inside a client block.

client block

The client block configures the HTTP client used by the component.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	The target URL to send telemetry data to.		yes
`encoding`	`string`	The encoding to use for messages. Should be either `"proto"` or `"json"`.	`"proto"`	no
`read_buffer_size`	`string`	Size of the read buffer the HTTP client uses for reading server responses.	`0`	no
`write_buffer_size`	`string`	Size of the write buffer the HTTP client uses for writing requests.	`"512KiB"`	no
`timeout`	`duration`	Time to wait before marking a request as failed.	`"30s"`	no
`headers`	`map(string)`	Additional headers to send with the request.	`{}`	no
`compression`	`string`	Compression mechanism to use for requests.	`"gzip"`	no
`max_idle_conns`	`int`	Limits the number of idle HTTP connections the client can keep open.	`100`	no
`max_idle_conns_per_host`	`int`	Limits the number of idle HTTP connections the host can keep open.	`0`	no
`max_conns_per_host`	`int`	Limits the total (dialing,active, and idle) number of connections per host.	`0`	no
`idle_conn_timeout`	`duration`	Time to wait before an idle connection closes itself.	`"90s"`	no
`disable_keep_alives`	`bool`	Disable HTTP keep-alive.	`false`	no
`http2_read_idle_timeout`	`duration`	Timeout after which a health check using ping frame will be carried out if no frame is received on the connection.	`0s`	no
`http2_ping_timeout`	`duration`	Timeout after which the connection will be closed if a response to Ping is not received.	`15s`	no
`auth`	`capsule(otelcol.Handler)`	Handler from an `otelcol.auth` component to use for authenticating requests.		no

When setting headers, note that:

Certain headers such as Content-Length and Connection are automatically written when needed and values in headers may be ignored.
The Host header is automatically derived from the endpoint value. However, this automatic assignment can be overridden by explicitly setting a Host header in headers.

Setting disable_keep_alives to true will result in significant overhead establishing a new HTTP(s) connection for every request. Before enabling this option, consider whether changes to idle connection settings can achieve your goal.

If http2_ping_timeout is unset or set to 0s, it will default to 15s.

If http2_read_idle_timeout is unset or set to 0s, then no health check will be performed.

By default, requests are compressed with gzip. The compression argument controls which compression mechanism to use. Supported strings are:

"gzip"
"zlib"
"deflate"
"snappy"
"zstd"

If compression is set to "none" or an empty string "", no compression is used.

tls block

The tls block configures TLS settings used for the connection to the HTTP server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

sending_queue block

The sending_queue block configures an in-memory buffer of batches before data is sent to the HTTP server.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables an in-memory buffer before sending data to the client.	`true`	no
`num_consumers`	`number`	Number of readers to send batches written to the queue in parallel.	`10`	no
`queue_size`	`number`	Maximum number of unwritten batches allowed in the queue at the same time.	`1000`	no

retry_on_failure block

The retry_on_failure block configures how failed requests to the HTTP server are retried.

The following arguments are supported:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enables retrying failed requests.	`true`	no
`initial_interval`	`duration`	Initial time to wait before retrying a failed request.	`"5s"`	no
`max_elapsed_time`	`duration`	Maximum time to wait before discarding a failed batch.	`"5m"`	no
`max_interval`	`duration`	Maximum time to wait between retries.	`"30s"`	no
`multiplier`	`number`	Factor to grow wait time before retrying.	`1.5`	no
`randomization_factor`	`number`	Factor to randomize wait time before retrying.	`0.5`	no

debug_metrics block

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

Name	Type	Description	Default	Required
`disable_high_cardinality_metrics`	`boolean`	Whether to disable certain high cardinality metrics.	`true`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.exporter.otlphttp is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.otlphttp does not expose any component-specific debug information.

Example

This example creates an exporter to send data to a locally running Grafana Tempo without TLS:

otelcol.exporter.otlphttp "tempo" {
    client {
        endpoint = "http://tempo:4317"
        tls {
            insecure             = true
            insecure_skip_verify = true
        }
    }
}

Compatible components

otelcol.exporter.otlphttp has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.exporter.prometheus

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.exporter.prometheus

otelcol.exporter.prometheus accepts OTLP-formatted metrics from other otelcol components, converts metrics to Prometheus-formatted metrics, and forwards the resulting metrics to prometheus components.

NOTE: otelcol.exporter.prometheus is a custom component unrelated to the prometheus exporter from OpenTelemetry Collector.

Conversion of metrics are done according to the OpenTelemetry Metrics Data Model specification.

Multiple otelcol.exporter.prometheus components can be specified by giving them different labels.

Usage

otelcol.exporter.prometheus "LABEL" {
  forward_to = [...]
}

Arguments

otelcol.exporter.prometheus supports the following arguments:

Name	Type	Description	Default	Required
`include_target_info`	`boolean`	Whether to include `target_info` metrics.	`true`	no
`include_scope_info`	`boolean`	Whether to include `otel_scope_info` metrics.	`false`	no
`include_scope_labels`	`boolean`	Whether to include additional OTLP labels in all metrics.	`true`	no
`add_metric_suffixes`	`boolean`	Whether to add type and unit suffixes to metrics names.	`true`	no
`gc_frequency`	`duration`	How often to clean up stale metrics from memory.	`"5m"`	no
`forward_to`	`list(MetricsReceiver)`	Where to forward converted Prometheus metrics.		yes
`resource_to_telemetry_conversion`	`boolean`	Whether to convert OTel resource attributes to Prometheus labels.	`false`	no

By default, OpenTelemetry resources are converted into target_info metrics. OpenTelemetry instrumentation scopes are converted into otel_scope_info metrics. Set the include_scope_info and include_target_info arguments to false, respectively, to disable the custom metrics.

When include_scope_labels is true the otel_scope_name and otel_scope_version labels are added to every converted metric sample.

When include_target_info is true, OpenTelemetry Collector resources are converted into target_info metrics.

Note
OTLP metrics can have a lot of resource attributes. Setting resource_to_telemetry_conversion to true would convert all of them to Prometheus labels, which may not be what you want. Instead of using resource_to_telemetry_conversion, most users need to use otelcol.processor.transform to convert OTLP resource attributes to OTLP metric datapoint attributes before using otelcol.exporter.prometheus. See Creating Prometheus labels from OTLP resource attributes for an example.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for metrics. Other telemetry signals are ignored.

Metrics sent to the input are converted to Prometheus-compatible metrics and are forwarded to the forward_to argument.

The following are dropped during the conversion process:

Metrics that use the delta aggregation temporality

Component health

otelcol.exporter.prometheus is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.exporter.prometheus does not expose any component-specific debug information.

Example

Basic usage

This example accepts metrics over OTLP and forwards it using prometheus.remote_write:

otelcol.receiver.otlp "default" {
  grpc {}

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Create Prometheus labels from OTLP resource attributes

This example uses otelcol.processor.transform to add extra key1 and key2 OTLP metric datapoint attributes from the key1 and key2 OTLP resource attributes.

otelcol.exporter.prometheus then converts key1 and key2 to Prometheus labels along with any other OTLP metric datapoint attributes.

This avoids the need to set resource_to_telemetry_conversion to true, which could have created too many unnecessary metric labels.

otelcol.receiver.otlp "default" {
  grpc {}

  output {
    metrics = [otelcol.processor.transform.default.input]
  }
}

otelcol.processor.transform "default" {
  error_mode = "ignore"

  metric_statements {
    context = "datapoint"

    statements = [
      `set(attributes["key1"], resource.attributes["key1"])`,
      `set(attributes["key2"], resource.attributes["key2"])`,
    ]
  }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Compatible components

otelcol.exporter.prometheus can accept arguments from the following components:

Components that export Prometheus MetricsReceiver

otelcol.exporter.prometheus has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.extension.jaeger_remote_sampling

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.extension.jaeger_remote_sampling

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.extension.jaeger_remote_sampling serves a specified Jaeger remote sampling document.

NOTE: otelcol.extension.jaeger_remote_sampling is a wrapper over the upstream OpenTelemetry Collector jaegerremotesampling extension. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.extension.jaeger_remote_sampling components can be specified by giving them different labels.

Usage

otelcol.extension.jaeger_remote_sampling "LABEL" {
  source {
  }
}

Arguments

otelcol.extension.jaeger_remote_sampling doesn’t support any arguments and is configured fully through inner blocks.

Blocks

The following blocks are supported inside the definition of otelcol.extension.jaeger_remote_sampling:

Hierarchy	Block	Description	Required
http	http	Configures the http server to serve Jaeger remote sampling.	no
http > tls	tls	Configures TLS for the HTTP server.	no
http > cors	cors	Configures CORS for the HTTP server.	no
grpc	grpc	Configures the grpc server to serve Jaeger remote sampling.	no
grpc > tls	tls	Configures TLS for the gRPC server.	no
grpc > keepalive	keepalive	Configures keepalive settings for the configured server.	no
grpc > keepalive > server_parameters	server_parameters	Server parameters used to configure keepalive settings.	no
grpc > keepalive > enforcement_policy	enforcement_policy	Enforcement policy for keepalive settings.	no
source	source	Configures the Jaeger remote sampling document.	yes
source > remote	remote	Configures the gRPC client used to retrieve the Jaeger remote sampling document.	no
source > remote > tls	tls	Configures TLS for the gRPC client.	no
source > remote > keepalive	keepalive	Configures keepalive settings for the gRPC client.	no

The > symbol indicates deeper levels of nesting. For example, grpc > tls refers to a tls block defined inside a grpc block.

http block

The http block configures an HTTP server which serves the Jaeger remote sampling document.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:5778"`	no
`max_request_body_size`	`string`	Maximum request body size the server will allow. No limit when unset.		no
`include_metadata`	`boolean`	Propagate incoming connection metadata to downstream consumers.		no

tls block

The tls block configures TLS settings used for a server. If the tls block isn’t provided, TLS won’t be used for connections to the server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

cors block

The cors block configures CORS settings for an HTTP server.

The following arguments are supported:

Name	Type	Description	Default	Required
`allowed_origins`	`list(string)`	Allowed values for the `Origin` header.		no
`allowed_headers`	`list(string)`	Accepted headers from CORS requests.	`["X-Requested-With"]`	no
`max_age`	`number`	Configures the `Access-Control-Max-Age` response header.		no

The allowed_headers specifies which headers are acceptable from a CORS request. The following headers are always implicitly allowed:

Accept
Accept-Language
Content-Type
Content-Language

If allowed_headers includes "*", all headers will be permitted.

grpc block

The grpc block configures a gRPC server which serves the Jaeger remote sampling document.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:14250"`	no
`transport`	`string`	Transport to use for the gRPC server.	`"tcp"`	no
`max_recv_msg_size`	`string`	Maximum size of messages the server will accept. 0 disables a limit.		no
`max_concurrent_streams`	`number`	Limit the number of concurrent streaming RPC calls.		no
`read_buffer_size`	`string`	Size of the read buffer the gRPC server will use for reading from clients.	`"512KiB"`	no
`write_buffer_size`	`string`	Size of the write buffer the gRPC server will use for writing to clients.		no
`include_metadata`	`boolean`	Propagate incoming connection metadata to downstream consumers.		no

keepalive block

The keepalive block configures keepalive settings for connections to a gRPC server.

keepalive doesn’t support any arguments and is configured fully through inner blocks.

server_parameters block

The server_parameters block controls keepalive and maximum age settings for gRPC servers.

The following arguments are supported:

Name	Type	Description	Default	Required
`max_connection_idle`	`duration`	Maximum age for idle connections.	`"infinity"`	no
`max_connection_age`	`duration`	Maximum age for non-idle connections.	`"infinity"`	no
`max_connection_age_grace`	`duration`	Time to wait before forcibly closing connections.	`"infinity"`	no
`time`	`duration`	How often to ping inactive clients to check for liveness.	`"2h"`	no
`timeout`	`duration`	Time to wait before closing inactive clients that do not respond to liveness checks.	`"20s"`	no

enforcement_policy block

The enforcement_policy block configures the keepalive enforcement policy for gRPC servers. The server will close connections from clients that violate the configured policy.

The following arguments are supported:

Name	Type	Description	Default	Required
`min_time`	`duration`	Minimum time clients should wait before sending a keepalive ping.	`"5m"`	no
`permit_without_stream`	`boolean`	Allow clients to send keepalive pings when there are no active streams.	`false`	no

source block

The source block configures the method of retrieving the Jaeger remote sampling document that is served by the servers specified in the grpc and http blocks.

The following arguments are supported:

Name	Type	Description	Default	Required
`file`	`string`	A local file containing a Jaeger remote sampling document.	`""`	no
`reload_interval`	`duration`	The interval at which to reload the specified file. Leave at 0 to never reload.	`0`	no
`content`	`string`	A string containing the Jaeger remote sampling contents directly.	`""`	no

Exactly one of the file argument, content argument or remote block must be specified.

remote block

The remote block configures the gRPC client used by the component.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to send telemetry data to.		yes
`compression`	`string`	Compression mechanism to use for requests.	`"gzip"`	no
`read_buffer_size`	`string`	Size of the read buffer the gRPC client to use for reading server responses.		no
`write_buffer_size`	`string`	Size of the write buffer the gRPC client to use for writing requests.	`"512KiB"`	no
`wait_for_ready`	`boolean`	Waits for gRPC connection to be in the `READY` state before sending data.	`false`	no
`headers`	`map(string)`	Additional headers to send with the request.	`{}`	no
`balancer_name`	`string`	Which gRPC client-side load balancer to use for requests.	`pick_first`	no
`authority`	`string`	Overrides the default `:authority` header in gRPC requests from the gRPC client.		no
`auth`	`capsule(otelcol.Handler)`	Handler from an `otelcol.auth` component to use for authenticating requests.		no

By default, requests are compressed with gzip. The compression argument controls which compression mechanism to use. Supported strings are:

"gzip"
"zlib"
"deflate"
"snappy"
"zstd"

If compression is set to "none" or an empty string "", no compression is used.

The supported values for balancer_name are listed in the gRPC documentation on Load balancing:

pick_first: Tries to connect to the first address, uses it for all RPCs if it connects, or tries the next address if it fails (and keeps doing that until one connection is successful). Because of this, all the RPCs will be sent to the same backend.
round_robin: Connects to all the addresses it sees and sends an RPC to each backend one at a time in order. For example, the first RPC is sent to backend-1, the second RPC is sent to backend-2, and the third RPC is sent to backend-1.

An HTTP proxy can be configured through the following environment variables:

HTTPS_PROXY
NO_PROXY

The HTTPS_PROXY environment variable specifies a URL to use for proxying requests. Connections to the proxy are established via the HTTP CONNECT method.

Because otelcol.extension.jaeger_remote_sampling uses gRPC, the configured proxy server must be able to handle and proxy HTTP/2 traffic.

tls client block

The tls block configures TLS settings used for the connection to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

keepalive client block

The keepalive block configures keepalive settings for gRPC client connections.

The following arguments are supported:

Name	Type	Description	Required
`ping_wait`	`duration`	How often to ping the server after no activity.	no
`ping_response_timeout`	`duration`	Time to wait before closing inactive connections if the server does not respond to a ping.	no
`ping_without_stream`	`boolean`	Send pings even if there is no active stream request.	no

Component health

otelcol.extension.jaeger_remote_sampling is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.extension.jaeger_remote_sampling does not expose any component-specific debug information.

Examples

Serving from a file

This example configures the Jaeger remote sampling extension to load a local json document and serve it over the default http port of 5778. Currently this config style exists for consistency with upstream Opentelemetry Collector components and may be removed.

otelcol.extension.jaeger_remote_sampling "example" {
  http {
  }
  source {
    file             = "/path/to/jaeger-sampling.json"
    reload_interval  = "10s"
  }
}

Serving from another component

This example uses the output of a component to determine what sampling rules to serve:

local.file "sampling" {
  filename  = "/path/to/jaeger-sampling.json"
}

otelcol.extension.jaeger_remote_sampling "example" {
  http {
  }
  source {
    content = local.file.sampling.content
  }
}

otelcol.processor.attributes

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.attributes

otelcol.processor.attributes accepts telemetry data from other otelcol components and modifies attributes of a span, log, or metric. It also supports the ability to filter and match input data to determine if it should be included or excluded for attribute modifications.

NOTE: otelcol.processor.attributes is a wrapper over the upstream OpenTelemetry Collector attributes processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

You can specify multiple otelcol.processor.attributes components by giving them different labels.

Usage

otelcol.processor.attributes "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.attributes doesn’t support any arguments and is configured fully through inner blocks.

Blocks

The following blocks are supported inside the definition of otelcol.processor.attributes:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes
action	action	Actions to take on the attributes of incoming metrics/logs/traces.	no
include	include	Filter for data included in this processor’s actions.	no
include > regexp	regexp	Regex cache settings.	no
include > attribute	attribute	A list of attributes to match against.	no
include > resource	resource	A list of items to match the resources against.	no
include > library	library	A list of items to match the implementation library against.	no
include > log_severity	library	How to match against a log record’s SeverityNumber, if defined.	no
exclude	exclude	Filter for data excluded from this processor’s actions	no
exclude > regexp	regexp	Regex cache settings.	no
exclude > attribute	attribute	A list of attributes to match against.	no
exclude > resource	resource	A list of items to match the resources against.	no
exclude > library	library	A list of items to match the implementation library against.	no
exclude > log_severity	log_severity	How to match against a log record’s SeverityNumber, if defined.	no

The > symbol indicates deeper levels of nesting. For example, include > attribute refers to an attribute block defined inside an include block.

If both an include block and an excludeblock are specified, the include properties are checked before the exclude properties.

action block

The action block configures how to modify the span, log, or metric.

The following attributes are supported:

Name	Type	Description	Default	Required
`key`	`string`	The attribute that the action relates to.		yes
`action`	`string`	The type of action performed.		yes
`value`	`any`	The value to populate for the key.		no
`pattern`	`string`	A regex pattern.	`""`	no
`from_attribute`	`string`	The attribute from the input data used to populate the attribute value.	`""`	no
`from_context`	`string`	The context value used to populate the attribute value.	`""`	no
`converted_type`	`string`	The type to convert the attribute value to.	`""`	no

The value data type must be either a number, string, or boolean.

The supported values for action are:

insert: Inserts a new attribute in input data where the key does not already exist.
- The key attribute is required. It specifies the attribute to act upon.
- One of the value, from_attribute or from_context attributes is required.
update: Updates an attribute in input data where the key does exist.
- The keyattribute is required. It specifies the attribute to act upon.
- One of the value, from_attribute or from_context attributes is required.
upsert: Either inserts a new attribute in input data where the key does not already exist or updates an attribute in input data where the key does exist.
- The keyattribute is required. It specifies the attribute to act upon.
- One of the value, from_attribute or from_contextattributes is required:
  - value specifies the value to populate for the key.
  - from_attribute specifies the attribute from the input data to use to populate the value. If the attribute doesn’t exist, no action is performed.
  - from_context specifies the context value used to populate the attribute value. If the key is prefixed with metadata., the values are searched in the receiver’s transport protocol for additional information like gRPC Metadata or HTTP Headers. If the key is prefixed with auth., the values are searched in the authentication information set by the server authenticator. Refer to the server authenticator’s documentation part of your pipeline for more information about which attributes are available. If the key doesn’t exist, no action is performed. If the key has multiple values the values will be joined with a ; separator.
hash: Hashes (SHA1) an existing attribute value.
- The key attribute and/or the pattern attributes is required.
extract: Extracts values using a regular expression rule from the input key to target keys specified in the rule. If a target key already exists, it will be overridden. Note: It behaves similarly to the Span Processor to_attributes setting with the existing attribute as the source.
- The key attribute is required. It specifies the attribute to extract values from. The value of key is NOT altered.
- The pattern attribute is required. It is the regex pattern used to extract attributes from the value of key. The submatchers must be named. If attributes already exist, they will be overwritten.
convert: Converts an existing attribute to a specified type.
- The key attribute is required. It specifies the attribute to act upon.
- The converted_type attribute is required and must be one of int, double or string.
delete: Deletes an attribute from the input data.
- The key attribute and/or the pattern attribute is required. It specifies the attribute to act upon.

include block

The include block provides an option to include data being fed into the action blocks based on the properties of a span, log, or metric records.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_type`	`string`	Controls how items to match against are interpreted.		yes
`log_bodies`	`list(string)`	A list of strings that the LogRecord’s body field must match against.	`[]`	no
`log_severity_texts`	`list(string)`	A list of strings that the LogRecord’s severity text field must match against.	`[]`	no
`metric_names`	`list(string)`	A list of strings to match the metric name against.	`[]`	no
`services`	`list(string)`	A list of items to match the service name against.	`[]`	no
`span_kinds`	`list(string)`	A list of items to match the span kind against.	`[]`	no
`span_names`	`list(string)`	A list of items to match the span name against.	`[]`	no

match_type is required and must be set to either "regexp" or "strict".

A match occurs if at least one item in the lists matches.

One of the following is also required:

For spans, one of services, span_names, span_kinds, attribute, resource, or library must be specified with a non-empty value for a valid configuration. The log_bodies, log_severity_texts, log_severity, and metric_names attributes are invalid.
For logs, one of log_bodies, log_severity_texts, log_severity, attribute, resource, or library must be specified with a non-empty value for a valid configuration. The span_names, span_kinds, metric_names, and services attributes are invalid.
For metrics, metric_names must be specified with a valid non-empty value for a valid configuration. The span_names, span_kinds, log_bodies, log_severity_texts, log_severity, services, attribute, resource, and library attributes are invalid.

If the configuration includes filters which are specific to a particular signal type, it is best to include only that signal type in the component’s output. For example, adding a span_names filter could cause the component to error if logs are configured in the component’s outputs.

exclude block

The exclude block provides an option to exclude data from being fed into the action blocks based on the properties of a span, log, or metric records.

Note
Signals excluded by the exclude block will still be propagated to downstream components as-is. If you would like to not propagate certain signals to downstream components, consider a processor such as otelcol.processor.tail_sampling.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_type`	`string`	Controls how items to match against are interpreted.		yes
`log_bodies`	`list(string)`	A list of strings that the LogRecord’s body field must match against.	`[]`	no
`log_severity_texts`	`list(string)`	A list of strings that the LogRecord’s severity text field must match against.	`[]`	no
`metric_names`	`list(string)`	A list of strings to match the metric name against.	`[]`	no
`services`	`list(string)`	A list of items to match the service name against.	`[]`	no
`span_kinds`	`list(string)`	A list of items to match the span kind against.	`[]`	no
`span_names`	`list(string)`	A list of items to match the span name against.	`[]`	no

match_type is required and must be set to either "regexp" or "strict".

A match occurs if at least one item in the lists matches.

One of the following is also required:

For spans, one of services, span_names, span_kinds, attribute, resource, or library must be specified with a non-empty value for a valid configuration. The log_bodies, log_severity_texts, log_severity, and metric_names attributes are invalid.
For logs, one of log_bodies, log_severity_texts, log_severity, attribute, resource, or library must be specified with a non-empty value for a valid configuration. The span_names, span_kinds, metric_names, and services attributes are invalid.
For metrics, metric_names must be specified with a valid non-empty value for a valid configuration. The span_names, span_kinds, log_bodies, log_severity_texts, log_severity, services, attribute, resource, and library attributes are invalid.

regexp block

This block is an optional configuration for the match_type of "regexp". It configures a Least Recently Used (LRU) cache.

The following arguments are supported:

Name	Type	Description	Default	Required
`cache_enabled`	`bool`	Determines whether match results are LRU cached.	`false`	no
`cache_max_num_entries`	`int`	The max number of entries of the LRU cache that stores match results.	`0`	no

Enabling cache_enabled could make subsequent matches faster. Cache size is unlimited unless cache_max_num_entries is also specified.

cache_max_num_entries is ignored if cache_enabled is false.

attribute block

This block specifies an attribute to match against:

More than one attribute block can be defined.
Only match_type = "strict" is allowed if attribute is specified.
All attribute blocks must match exactly for a match to occur.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	The attribute key.		yes
`value`	`any`	The attribute value to match against.		no

If value isn’t set, any value will match. The type of value could be a number, a string, or a boolean.

resource block

This block specifies items to match the resources against:

More than one resource block can be defined.
A match occurs if the input data resources match at least one resource block.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	The resource key.		yes
`value`	`any`	The resource value to match against.		no

If value is not set, any value will match. The type of value could be a number, a string, or a boolean.

library block

This block specifies properties to match the implementation library against:

More than one library block can be defined.
A match occurs if the span’s implementation library matches at least one library block.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The attribute key.		yes
`version`	`string`	The version to match against.	null	no

If version is unset, any version matches. If version is set to an empty string, it only matches a library version, which is also an empty string.

log_severity block

This block defines how to match based on a log record’s SeverityNumber field.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_undefined`	`bool`	Whether logs with “undefined” severity match.		yes
`min`	`string`	The lowest severity that may be matched.		yes

If match_undefined is true, entries with undefined severity will match.

The following table lists the severities supported by OTel. The value for min should be one of the values in the “Log Severity” column.

Log Severity	Severity number
TRACE	1
TRACE2	2
TRACE3	3
TRACE4	4
DEBUG	5
DEBUG2	6
DEBUG3	7
DEBUG4	8
INFO	9
INFO2	10
INFO3	11
INFO4	12
WARN	13
WARN2	14
WARN3	15
WARN4	16
ERROR	17
ERROR2	18
ERROR3	19
ERROR4	20
FATAL	21
FATAL2	22
FATAL3	23
FATAL4	24

For example, if the min attribute in the log_severity block is “INFO”, then INFO, WARN, ERROR, and FATAL logs will match.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the metrics, logs, and traces arguments accordingly to send telemetry data to other components.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.attributes is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.attributes does not expose any component-specific debug information.

Examples

Various uses of the “action” block

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.attributes.default.input]
    logs    = [otelcol.processor.attributes.default.input]
    traces  = [otelcol.processor.attributes.default.input]
  }
}

otelcol.processor.attributes "default" {
    // Inserts a new attribute "attribute1" to spans where
    // the key "attribute1" doesn't exist.
    // The type of `attribute1` is inferred by the configuration.
    // `123` is an integer and is stored as an integer in the attributes.
    action {
        key = "attribute1"
        value = 123
        action = "insert"
    }

    // Inserts a new attribute with a key of "string key" and
    // a string value of "anotherkey".
    action {
        key = "string key"
        value = "anotherkey"
        action = "insert"
    }

    // Setting an attribute on all spans.
    // Any spans that already had `region` now have value `planet-earth`.
    // This can be done to set properties for all traces without
    // requiring an instrumentation change.
    action {
        key = "region"
        value = "planet-earth"
        action = "upsert"
    }

    // The following demonstrates copying a value to a new key.
    // If a span doesn't contain `user_key`, no new attribute `new_user_key` is created.
    action {
        key = "new_user_key"
        from_attribute = "user_key"
        action = "upsert"
    }

    // Hashing existing attribute values.
    action {
        key = "user.email"
        action = "hash"
    }

    // Uses the value from key `example_user_key` to upsert attributes
    // to the target keys specified in the `pattern`.
    // (Insert attributes for target keys that do not exist and update keys that exist.)
    // Given example_user_key = /api/v1/document/12345678/update/v1
    // then the following attributes will be inserted:
    // new_example_user_key: 12345678
    // version: v1
    //
    // Note: Similar to the Span Processor, if a target key already exists,
    // it will be updated.
    //
    // Note: The regex pattern is enclosed in backticks instead of quotation marks.
    // This constitutes a raw River string, and lets us avoid the need to escape backslash characters.
    action {
        key = "example_user_key"
        pattern = `\/api\/v1\/document\/(?P<new_user_key>.*)\/update\/(?P<version>.*)$`
        action = "extract"
    }

    // Converting the type of an existing attribute value.
    action {
        key = "http.status_code"
        converted_type = "int"
        action = "convert"
    }

    // Deleting keys from an attribute.
    action {
        key = "credit_card"
        action = "delete"
    }

    output {
        metrics = [otelcol.exporter.otlp.default.input]
        logs    = [otelcol.exporter.otlp.default.input]
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = env("OTLP_ENDPOINT")
  }
}

Excluding spans based on attributes

For example, the following spans match the properties and won’t be processed by the processor:

Span1 Name: “svcB”, Attributes: {env: “dev”, test_request: 123, credit_card: 1234}
Span2 Name: “svcA”, Attributes: {env: “dev”, test_request: false}

The following spans do not match the properties and the processor actions are applied to it:

Span3 Name: “svcB”, Attributes: {env: 1, test_request: “dev”, credit_card: 1234}
Span4 Name: “svcC”, Attributes: {env: “dev”, test_request: false}

Note that due to the presence of the services attribute, this configuration works only for trace signals. This is why only traces are configured in the output block.

otelcol.processor.attributes "default" {
    exclude {
        match_type = "strict"
        services = ["svcA", "svcB"]
        attribute {
            key = "env"
            value = "dev"
        }
        attribute {
            key = "test_request"
        }
    }
    action {
        key = "credit_card"
        action = "delete"
    }
    action {
        key = "duplicate_key"
        action = "delete"
    }
    output {
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

Excluding spans based on resources

A “strict” match_type means that we must strictly match the resource key/value pairs.

Note that the resource attribute is not used for metrics, which is why metrics are not configured in the component output.

otelcol.processor.attributes "default" {
    exclude {
        match_type = "strict"
        resource {
            key = "host.type"
            value = "n1-standard-1"
        }
    }
    action {
        key = "credit_card"
        action = "delete"
    }
    action {
        key = "duplicate_key"
        action = "delete"
    }
    output {
        logs    = [otelcol.exporter.otlp.default.input]
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

Excluding spans based on a specific library version

A “strict” match_type means that we must strictly match the library key/value pairs.

Note that the library attribute is not used for metrics, which is why metrics are not configured in the component output.

otelcol.processor.attributes "default" {
    exclude {
        match_type = "strict"
        library {
            name = "mongo-java-driver"
            version = "3.8.0"
        }
    }
    action {
        key = "credit_card"
        action = "delete"
    }
    action {
        key = "duplicate_key"
        action = "delete"
    }
    output {
        logs    = [otelcol.exporter.otlp.default.input]
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

Including and excluding spans based on regex and services

This processor will remove the “token” attribute and will obfuscate the “password” attribute in spans where the service name matches "auth.*" and where the span name does not match "login.*".

Note that due to the presence of the services and span_names attributes, this configuration works only for trace signals. This is why only traces are configured in the output block.

otelcol.processor.attributes "default" {
    // Specifies the span properties that must exist for the processor to be applied.
    include {
        // "match_type" defines that "services" is an array of regexp-es.
        match_type = "regexp"
        // The span service name must match "auth.*" pattern.
        services = ["auth.*"]
    }

    exclude {
        // "match_type" defines that "span_names" is an array of regexp-es.
        match_type = "regexp"
        // The span name must not match "login.*" pattern.
        span_names = ["login.*"]
    }

    action {
        key = "password"
        action = "update"
        value = "obfuscated"
    }

    action {
        key = "token"
        action = "delete"
    }

    output {
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

Including spans based on regex and attributes

The following demonstrates how to process spans with attributes that match a regexp pattern. This processor will obfuscate the “db.statement” attribute in spans where the “db.statement” attribute matches a regex pattern.

otelcol.processor.attributes "default" {
    include {
        // "match_type" of "regexp" defines that the "value" attributes 
        // in the "attribute" blocks are regexp-es.
        match_type = "regexp"

        // This attribute ('db.statement') must exist in the span and match 
        // the regex ('SELECT \* FROM USERS.*') for a match.
        attribute {
            key = "db.statement"
            value = "SELECT \* FROM USERS.*"
        }
    }

    action {
        key = "db.statement"
        action = "update"
        value = "SELECT * FROM USERS [obfuscated]"
    }

    output {
        metrics = [otelcol.exporter.otlp.default.input]
        logs    = [otelcol.exporter.otlp.default.input]
        traces  = [otelcol.exporter.otlp.default.input]
    }
}

Including spans based on regex of log body

This processor will remove the “token” attribute and will obfuscate the “password” attribute in spans where the log body matches “AUTH.*”.

Note that due to the presence of the log_bodies attribute, this configuration works only for log signals. This is why only logs are configured in the output block.

otelcol.processor.attributes "default" {
    include {
        match_type = "regexp"
        log_bodies = ["AUTH.*"]
    }
    action {
        key = "password"
        action = "update"
        value = "obfuscated"
    }
    action {
        key = "token"
        action = "delete"
    }

    output {
        logs    = [otelcol.exporter.otlp.default.input]
    }
}

Including spans based on regex of log severity

The following demonstrates how to process logs that have a severity level which is equal to or higher than the level specified in the log_severity block. This processor will remove the “token” attribute and will obfuscate the “password” attribute in logs where the severity is at least “INFO”.

Note that due to the presence of the log_severity attribute, this configuration works only for log signals. This is why only logs are configured in the output block.

otelcol.processor.attributes "default" {
    include {
        match_type = "regexp"
		log_severity {
			min = "INFO"
			match_undefined = true
		}
    }
    action {
        key = "password"
        action = "update"
        value = "obfuscated"
    }
    action {
        key = "token"
        action = "delete"
    }

    output {
        logs    = [otelcol.exporter.otlp.default.input]
    }
}

Including spans based on regex of log severity text

The following demonstrates how to process logs that have a severity text that match regexp patterns. This processor will remove the “token” attribute and will obfuscate the “password” attribute in logs where severity matches “info”.

Note that due to the presence of the log_severity_texts attribute, this configuration works only for log signals. This is why only logs are configured in the output block.

otelcol.processor.attributes "default" {
    include {
        match_type = "regexp"
        log_severity_texts = ["info.*"]
    }
    action {
        key = "password"
        action = "update"
        value = "obfuscated"
    }
    action {
        key = "token"
        action = "delete"
    }

    output {
        logs    = [otelcol.exporter.otlp.default.input]
    }
}

Including metrics based on metric names

The following demonstrates how to process metrics that have a name starting with “counter”. This processor will add a label called “important_label” with a value of “label_val” to the metric. If the label already exists, its value will be updated.

Note that due to the presence of the metric_names attribute, this configuration works only for metric signals. This is why only metrics are configured in the output block.

otelcol.processor.attributes "default" {
	include {
		match_type = "regexp"
		metric_names = ["counter.*"]
	}
	action {
		key = "important_label"
		action = "upsert"
		value = "label_val"
	}

    output {
        metrics = [otelcol.exporter.otlp.default.input]
    }
}

Compatible components

otelcol.processor.attributes can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.attributes has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.batch

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.batch

otelcol.processor.batch accepts telemetry data from other otelcol components and places them into batches. Batching improves the compression of data and reduces the number of outgoing network requests required to transmit data. This processor supports both size and time based batching.

We strongly recommend that you configure the batch processor on every Agent that uses OpenTelemetry (otelcol) Flow components. The batch processor should be defined in the pipeline after the otelcol.processor.memory_limiter as well as any sampling processors. This is because batching should happen after any data drops such as sampling.

NOTE: otelcol.processor.batch is a wrapper over the upstream OpenTelemetry Collector batch processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.processor.batch components can be specified by giving them different labels.

Usage

otelcol.processor.batch "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.batch supports the following arguments:

Name	Type	Description	Default	Required
`timeout`	`duration`	How long to wait before flushing the batch.	`"200ms"`	no
`send_batch_size`	`number`	Amount of data to buffer before flushing the batch.	`8192`	no
`send_batch_max_size`	`number`	Upper limit of a batch size.	`0`	no
`metadata_keys`	`list(string)`	Creates a different batcher for each key/value combination of metadata.	`[]`	no
`metadata_cardinality_limit`	`number`	Limit of the unique metadata key/value combinations.	`1000`	no

otelcol.processor.batch accumulates data into a batch until one of the following events happens:

The duration specified by timeout elapses since the time the last batch was sent.
The number of spans, log lines, or metric samples processed is greater than or equal to the number specified by send_batch_size.

Logs, traces, and metrics are processed independently. For example, if send_batch_size is set to 1000:

The processor may, at the same time, buffer 1,000 spans, 1,000 log lines, and 1,000 metric samples before flushing them.
If there are enough spans for a batch of spans (1,000 or more), but not enough for a batch of metric samples (less than 1,000) then only the spans will be flushed.

Use send_batch_max_size to limit the amount of data contained in a single batch:

When set to 0, batches can be any size.
When set to a non-zero value, send_batch_max_size must be greater than or equal to send_batch_size. Every batch will contain up to the send_batch_max_size number of spans, log lines, or metric samples. The excess spans, log lines, or metric samples will not be lost - instead, they will be added to the next batch.

For example, assume send_batch_size is set to the default 8192 and there are currently 8,000 batched spans. If the batch processor receives 8,000 more spans at once, its behavior depends on how send_batch_max_size is configured:

If send_batch_max_size is set to 0, the total batch size would be 16,000 which would then be flushed as a single batch.
If send_batch_max_size is set to 10000, then the total batch size will be 10,000 and the remaining 6,000 spans will be flushed in a subsequent batch.

metadata_cardinality_limit applies for the lifetime of the process.

Receivers should be configured with include_metadata = true so that metadata keys are available to the processor.

Each distinct combination of metadata triggers the allocation of a new background task in the Agent that runs for the lifetime of the process, and each background task holds one pending batch of up to send_batch_size records. Batching by metadata can therefore substantially increase the amount of memory dedicated to batching.

The maximum number of distinct combinations is limited to the configured metadata_cardinality_limit, which defaults to 1000 to limit memory impact.

Blocks

The following blocks are supported inside the definition of otelcol.processor.batch:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.batch is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.batch does not expose any component-specific debug information.

Debug metrics

processor_batch_batch_send_size_ratio (histogram): Number of units in the batch.
processor_batch_metadata_cardinality_ratio (gauge): Number of distinct metadata value combinations being processed.
processor_batch_timeout_trigger_send_ratio_total (counter): Number of times the batch was sent due to a timeout trigger.
processor_batch_batch_size_trigger_send_ratio_total (counter): Number of times the batch was sent due to a size trigger.

Examples

Basic usage

This example batches telemetry data before sending it to otelcol.exporter.otlp for further processing:

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.production.input]
    logs    = [otelcol.exporter.otlp.production.input]
    traces  = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
  }
}

Batching with a timeout

This example will buffer up to 10,000 spans, metric data points, or log records for up to 10 seconds. Because send_batch_max_size is not set, the batch size may exceed 10,000.

otelcol.processor.batch "default" {
  timeout = "10s"
  send_batch_size = 10000

  output {
    metrics = [otelcol.exporter.otlp.production.input]
    logs    = [otelcol.exporter.otlp.production.input]
    traces  = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
  }
}

Batching based on metadata

Batching by metadata enables support for multi-tenant OpenTelemetry pipelines with batching over groups of data having the same authorization metadata.

otelcol.receiver.jaeger "default" {
  protocols {
    grpc {
      include_metadata = true
    }
    thrift_http {}
    thrift_binary {}
    thrift_compact {}
  }

  output {
    traces = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  // batch data by tenant id
  metadata_keys = ["tenant_id"]
  // limit to 10 batcher processes before raising errors
  metadata_cardinality_limit = 123

  output {
    traces  = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
  }
}

Compatible components

otelcol.processor.batch can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.batch has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.discovery

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.discovery

otelcol.processor.discovery accepts traces telemetry data from other otelcol components. It can be paired with discovery.* components, which supply a list of labels for each discovered target. otelcol.processor.discovery adds resource attributes to spans which have a hostname matching the one in the __address__ label provided by the discovery.* component.

Note
otelcol.processor.discovery is a custom component unrelated to any processors from the OpenTelemetry Collector.

Multiple otelcol.processor.discovery components can be specified by giving them different labels.

Note
It can be difficult to follow OpenTelemetry semantic conventions when adding resource attributes via otelcol.processor.discovery:

discovery.relabel and most discovery.* processes such as discovery.kubernetes can only emit Prometheus-compatible labels.

Prometheus labels use underscores (_) in labels names, whereas OpenTelemetry semantic conventions use dots (.).

Although otelcol.processor.discovery is able to work with non-Prometheus labels such as ones containing dots, the fact that discovery.* components are generally only compatible with Prometheus naming conventions makes it hard to follow OpenTelemetry semantic conventions in otelcol.processor.discovery.

If your use case is to add resource attributes which contain Kubernetes metadata, consider using otelcol.processor.k8sattributes instead.

The main use case for otelcol.processor.discovery is for users who migrate to Grafana Agent Flow from Static mode’s prom_sd_operation_type/prom_sd_pod_associations configuration options.

Usage

otelcol.processor.discovery "LABEL" {
  targets = [...]
  output {
    traces  = [...]
  }
}

Arguments

otelcol.processor.discovery supports the following arguments:

Name	Type	Description	Default	Required
`targets`	`list(map(string))`	List of target labels to apply to the spans.		yes
`operation_type`	`string`	Configures whether to update a span’s attribute if it already exists.	`upsert`	no
`pod_associations`	`list(string)`	Configures how to decide the hostname of the span.	`["ip", "net.host.ip", "k8s.pod.ip", "hostname", "connection"]`	no

targets could come from discovery.* components:

The __address__ label will be matched against the IP address of incoming spans.
- If __address__ contains a port, it is ignored.
If a match is found, then relabeling rules are applied.
- Note that labels starting with __ will not be added to the spans.

The supported values for operation_type are:

insert: Inserts a new resource attribute if the key does not already exist.
update: Updates a resource attribute if the key already exists.
upsert: Either inserts a new resource attribute if the key does not already exist, or updates a resource attribute if the key does exist.

The supported values for pod_associations are:

ip: The hostname will be sourced from an ip resource attribute.
net.host.ip: The hostname will be sourced from a net.host.ip resource attribute.
k8s.pod.ip: The hostname will be sourced from a k8s.pod.ip resource attribute.
hostname: The hostname will be sourced from a host.name resource attribute.
connection: The hostname will be sourced from the context from the incoming requests (gRPC and HTTP).

If multiple pod_associations methods are enabled, the order of evaluation is honored. For example, when pod_associations is ["ip", "net.host.ip"], "net.host.ip" may be matched only if "ip" has not already matched.

Blocks

The following blocks are supported inside the definition of otelcol.processor.discovery:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the traces argument accordingly to send telemetry data to other components.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer OTLP-formatted data for telemetry signals of these types:

traces

Component health

otelcol.processor.discovery is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.discovery does not expose any component-specific debug information.

Examples

Basic usage

discovery.http "dynamic_targets" {
    url              = "https://example.com/scrape_targets"
    refresh_interval = "15s"
}

otelcol.processor.discovery "default" {
    targets = discovery.http.dynamic_targets.targets

    output {
        traces = [otelcol.exporter.otlp.default.input]
    }
}

Using more than one discovery process

Outputs from more than one discovery process can be combined via the concat function.

discovery.http "dynamic_targets" {
    url              = "https://example.com/scrape_targets"
    refresh_interval = "15s"
}

discovery.kubelet "k8s_pods" {
  bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
  namespaces        = ["default", "kube-system"]
}

otelcol.processor.discovery "default" {
    targets = concat(discovery.http.dynamic_targets.targets, discovery.kubelet.k8s_pods.targets)

    output {
        traces = [otelcol.exporter.otlp.default.input]
    }
}

Using a preconfigured list of attributes

It is not necessary to use a discovery component. In the example below, both a test_label and a test.label.with.dots resource attributes will be added to a span if its IP address is “1.2.2.2”. The __internal_label__ will be not be added to the span, because it begins with a double underscore (__).

otelcol.processor.discovery "default" {
    targets = [{
        "__address__"          = "1.2.2.2", 
        "__internal_label__"   = "test_val",
        "test_label"           = "test_val2",
        "test.label.with.dots" = "test.val2.with.dots"}]

    output {
        traces = [otelcol.exporter.otlp.default.input]
    }
}

Compatible components

otelcol.processor.discovery can accept arguments from the following components:

Components that export Targets
Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.discovery has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.filter

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.filter

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.processor.filter accepts and filters telemetry data from other otelcol components using the OpenTelemetry Transformation Language (OTTL). If any of the OTTL statements evaluates to true, the telemetry data is dropped.

OTTL statements consist of OTTL Converter functions, which act on paths. A path is a reference to a telemetry data such as:

Resource attributes.
Instrumentation scope name.
Span attributes.

In addition to the standard OTTL Converter functions, the following metrics-only functions are used exclusively by the processor:

OTTL statements used in otelcol.processor.filter mostly contain constructs such as:

Booleans:
- not true
- not IsMatch(name, "http_.*")
Math expressions:
- 1 + 1
- end_time_unix_nano - start_time_unix_nano
- sum([1, 2, 3, 4]) + (10 / 1) - 1

Note
Raw River strings can be used to write OTTL statements. For example, the OTTL statement attributes["grpc"] == true is written in River as `attributes[“grpc”] == true`

Note
otelcol.processor.filter is a wrapper over the upstream OpenTelemetry Collector filter processor. If necessary, bug reports or feature requests will be redirected to the upstream repository.

You can specify multiple otelcol.processor.filter components by giving them different labels.

Warning
Exercise caution when using otelcol.processor.filter:

Make sure you understand schema/format of the incoming data and test the configuration thoroughly. In general, use a configuration that is as specific as possible ensure you retain only the data you want to keep.

Orphaned Telemetry: The processor allows dropping spans. Dropping a span may lead to orphaned spans if the dropped span is a parent. Dropping a span may lead to orphaned logs if the log references the dropped span.

Usage

otelcol.processor.filter "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.filter supports the following arguments:

Name	Type	Description	Default	Required
`error_mode`	`string`	How to react to errors if they occur while processing a statement.	`"propagate"`	no

The supported values for error_mode are:

ignore: Ignore errors returned by conditions, log them, and continue on to the next condition. This is the recommended mode.
silent: Ignore errors returned by conditions, do not log them, and continue on to the next condition.
propagate: Return the error up the pipeline. This will result in the payload being dropped from Grafana Agent.

Blocks

The following blocks are supported inside the definition of otelcol.processor.filter:

Hierarchy	Block	Description	Required
traces	traces	Statements which filter traces.	no
metrics	metrics	Statements which filter metrics.	no
logs	logs	Statements which filter logs.	no
output	output	Configures where to send received telemetry data.	yes

traces block

The traces block specifies statements that filter trace telemetry signals. Only one traces block can be specified.

Name	Type	Description	Default	Required
`span`	`list(string)`	List of OTTL statements filtering OTLP spans.		no
`spanevent`	`list(string)`	List of OTTL statements filtering OTLP span events.		no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

Statements are checked in order from “high level” to “low level” telemetry, in this order:

span
spanevent

If at least one span condition is satisfied, the spanevent conditions will not be checked. Only one of the statements inside the list of statements has to be satisfied.

If all span events for a span are dropped, the span will be left intact.

metrics block

The metrics block specifies statements that filter metric telemetry signals. Only one metrics blocks can be specified.

Name	Type	Description	Default	Required
`metric`	`list(string)`	List of OTTL statements filtering OTLP metric.		no
`datapoint`	`list(string)`	List of OTTL statements filtering OTLP metric datapoints.		no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

Statements are checked in order from “high level” to “low level” telemetry, in this order:

metric
datapoint

If at least one metric condition is satisfied, the datapoint conditions will not be checked. Only one of the statements inside the list of statements has to be satisfied.

If all datapoints for a metric are dropped, the metric will also be dropped.

logs block

The logs block specifies statements that filter log telemetry signals. Only logs blocks can be specified.

Name	Type	Description	Default	Required
`log_record`	`list(string)`	List of OTTL statements filtering OTLP metric.		no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

OTTL log context

Only one of the statements inside the list of statements has to be satisfied.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.filter is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.filter does not expose any component-specific debug information.

Debug metrics

otelcol.processor.filter does not expose any component-specific debug metrics.

Examples

Drop spans which contain a certain span attribute

This example sets the attribute test to pass if the attribute test does not exist.

otelcol.processor.filter "default" {
  error_mode = "ignore"

  traces {
    span = [
      "attributes[\"container.name\"] == \"app_container_1\"",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each " is escaped with \" inside the River string.

Drop metrics based on either of two criteria

This example drops metrics which satisfy at least one of two OTTL statements:

The metric name is my.metric and there is a my_label resource attribute with a value of abc123 .
The metric is a histogram.

otelcol.processor.filter "default" {
  error_mode = "ignore"

  metrics {
    metric = [
       "name == \"my.metric\" and resource.attributes[\"my_label\"] == \"abc123\""
       "type == METRIC_DATA_TYPE_HISTOGRAM"
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Some values in the River string are escaped:

\ is escaped with \\
" is escaped with \"

Drop non-HTTP spans and sensitive logs

otelcol.processor.filter "default" {
  error_mode = "ignore"

  traces {
    span = [
      "attributes[\"http.request.method\"] == nil",
    ]
  }

  logs {
    log_record = [
      "IsMatch(body, \".*password.*\")",
      "severity_number < SEVERITY_NUMBER_WARN",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each " is escaped with \" inside the River string.

Some values in the River strings are escaped:

\ is escaped with \\
" is escaped with \"

Compatible components

otelcol.processor.filter can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.filter has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.k8sattributes

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.k8sattributes

otelcol.processor.k8sattributes accepts telemetry data from other otelcol components and adds Kubernetes metadata to the resource attributes of spans, logs, or metrics.

Note
otelcol.processor.k8sattributes is a wrapper over the upstream OpenTelemetry Collector k8sattributes processor. If necessary, bug reports or feature requests will be redirected to the upstream repository.

You can specify multiple otelcol.processor.k8sattributes components by giving them different labels.

Usage

otelcol.processor.k8sattributes "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

The following arguments are supported:

Name	Type	Description	Default	Required
`auth_type`	`string`	Authentication method when connecting to the Kubernetes API.	`serviceAccount`	no
`passthrough`	`bool`	Passthrough signals as-is, only adding a `k8s.pod.ip` resource attribute.	`false`	no

The supported values for auth_type are:

none: No authentication is required.
serviceAccount: Use the built-in service account that Kubernetes automatically provisions for each pod.
kubeConfig: Use local credentials like those used by kubectl.
tls: Use client TLS authentication.

Setting passthrough to true enables the “passthrough mode” of otelcol.processor.k8sattributes:

Only a k8s.pod.ip resource attribute will be added.
No other metadata will be added.
The Kubernetes API will not be accessed.
To correctly detect the pod IPs, Grafana Agent must receive spans directly from services.
The passthrough setting is useful when configuring the Agent as a Kubernetes Deployment. A Grafana Agent running as a Deployment cannot detect the IP addresses of pods generating telemetry data without any of the well-known IP attributes. If the Deployment Grafana Agent receives telemetry from Grafana Agents deployed as DaemonSet, then some of those attributes might be missing. As a workaround, you can configure the DaemonSet Grafana Agents with passthrough set to true.

Blocks

The following blocks are supported inside the definition of otelcol.processor.k8sattributes:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes
extract	extract	Rules for extracting data from Kubernetes.	no
extract > annotation	annotation	Creating resource attributes from Kubernetes annotations.	no
extract > label	extract_label	Creating resource attributes from Kubernetes labels.	no
filter	filter	Filters the data loaded from Kubernetes.	no
filter > field	field	Filter pods by generic Kubernetes fields.	no
filter > label	filter_label	Filter pods by Kubernetes labels.	no
pod_association	pod_association	Rules to associate pod metadata with telemetry signals.	no
pod_association > source	source	Source information to identify a pod.	no
exclude	exclude	Exclude pods from being processed.	no
exclude > pod	pod	Pod information.	no

The > symbol indicates deeper levels of nesting. For example, extract > annotation refers to an annotation block defined inside an extract block.

extract block

The extract block configures which metadata, annotations, and labels to extract from the pod.

The following attributes are supported:

Name	Type	Description	Default	Required
`metadata`	`list(string)`	Pre-configured metadata keys to add.	See below	no

The currently supported metadata keys are:

k8s.pod.name
k8s.pod.uid
k8s.deployment.name
k8s.node.name
k8s.namespace.name
k8s.pod.start_time
k8s.replicaset.name
k8s.replicaset.uid
k8s.daemonset.name
k8s.daemonset.uid
k8s.job.name
k8s.job.uid
k8s.cronjob.name
k8s.statefulset.name
k8s.statefulset.uid
k8s.container.name
container.image.name
container.image.tag
container.id

By default, if metadata is not specified, the following fields are extracted and added to spans, metrics, and logs as resource attributes:

k8s.pod.name
k8s.pod.uid
k8s.pod.start_time
k8s.namespace.name
k8s.node.name
k8s.deployment.name (if the pod is controlled by a deployment)
k8s.container.name (requires an additional attribute to be set: container.id)
container.image.name (requires one of the following additional attributes to be set: container.id or k8s.container.name)
container.image.tag (requires one of the following additional attributes to be set: container.id or k8s.container.name)

annotation block

The annotation block configures how to extract Kubernetes annotations.

The following attributes are supported:

Name	Type	Description	Default	Required
`from`	`string`	The source of the labels or annotations. Allowed values are `pod`, `namespace`, and `node`.	`pod`	no
`key_regex`	`string`	A regular expression used to extract a key that matches the regular expression.	`""`	no
`key`	`string`	The annotation or label name. This key must exactly match an annotation or label name.	`""`	no
`regex`	`string`	An optional field used to extract a sub-string from a complex field value.	`""`	no
`tag_name`	`string`	The name of the resource attribute added to logs, metrics, or spans.	`""`	no

When you don’t specify the tag_name, a default tag name is used with the format:

k8s.pod.annotations.<annotation key>
k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your pod spec contains the following labels:

app.kubernetes.io/component: mysql
app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

extract {
	label {
	    from = "pod"
		key_regex = "kubernetes.io/(.*)"
		tag_name  = "$1"
	}
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to either "pod" or "namespace".

label block

The label block configures how to extract Kubernetes labels.

The following attributes are supported:

Name	Type	Description	Default	Required
`from`	`string`	The source of the labels or annotations. Allowed values are `pod`, `namespace`, and `node`.	`pod`	no
`key_regex`	`string`	A regular expression used to extract a key that matches the regular expression.	`""`	no
`key`	`string`	The annotation or label name. This key must exactly match an annotation or label name.	`""`	no
`regex`	`string`	An optional field used to extract a sub-string from a complex field value.	`""`	no
`tag_name`	`string`	The name of the resource attribute added to logs, metrics, or spans.	`""`	no

When you don’t specify the tag_name, a default tag name is used with the format:

k8s.pod.annotations.<annotation key>
k8s.pod.labels.<label key>

For example, if tag_name isn’t specified and the key is git_sha, the attribute name will be k8s.pod.annotations.git_sha.

You can set either the key attribute or the key_regex attribute, but not both. When key_regex is present, tag_name supports back reference to both named capturing and positioned capturing.

For example, assume your pod spec contains the following labels:

app.kubernetes.io/component: mysql
app.kubernetes.io/version: 5.7.21

If you’d like to add tags for all labels with the prefix app.kubernetes.io/ and trim the prefix, then you can specify the following extraction rules:

extract {
	label {
	    from = "pod"
		key_regex = "kubernetes.io/(.*)"
		tag_name  = "$1"
	}
}

These rules add the component and version tags to the spans or metrics.

You can set the from attribute to either "pod" or "namespace".

The filter block configures which nodes to get data from and which fields and labels to fetch.

The following attributes are supported:

Name	Type	Description	Default	Required
`node`	`string`	Configures a Kubernetes node name or host name.	`""`	no
`namespace`	`string`	Filters all pods by the provided namespace. All other pods are ignored.	`""`	no

If node is specified, then any pods not running on the specified node will be ignored by otelcol.processor.k8sattributes.

field block

The field block allows you to filter pods by generic Kubernetes fields.

The following attributes are supported:

Name	Type	Description	Default	Required
`key`	`string`	The key or name of the field or labels that a filter can use.		yes
`value`	`string`	The value associated with the key that a filter can use.		yes
`op`	`string`	The filter operation to apply on the given key: value pair.	`equals`	no

For op, the following values are allowed:

equals: The field value must equal the provided value.
not-equals: The field value must not be equal to the provided value.
exists: The field value must exist. Only applicable to annotation fields.
does-not-exist: The field value must not exist. Only applicable to annotation fields.

The label block allows you to filter pods by generic Kubernetes labels.

The following attributes are supported:

Name	Type	Description	Default	Required
`key`	`string`	The key or name of the field or labels that a filter can use.		yes
`value`	`string`	The value associated with the key that a filter can use.		yes
`op`	`string`	The filter operation to apply on the given key: value pair.	`equals`	no

For op, the following values are allowed:

equals: The field value must equal the provided value.
not-equals: The field value must not be equal to the provided value.
exists: The field value must exist. Only applicable to annotation fields.
does-not-exist: The field value must not exist. Only applicable to annotation fields.

pod_association block

The pod_association block configures rules on how to associate logs/traces/metrics to pods.

The pod_association block does not support any arguments and is configured fully through child blocks.

The pod_association block can be repeated multiple times, to configure additional rules.

Example:

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.ip"
    }
}

pod_association {
    source {
        from = "resource_attribute"
        name = "k8s.pod.uid"
    }
    source {
        from = "connection"
    }
}

source block

The source block configures a pod association rule. This is used by the k8sattributes processor to determine the pod associated with a telemetry signal.

When multiple source blocks are specified inside a pod_association block, both source blocks has to match for the pod to be associated with the telemetry signal.

The following attributes are supported:

Name	Type	Description	Default	Required
`from`	`string`	The association method. Currently supports `resource_attribute` and `connection`		yes
`name`	`string`	Name represents extracted key name. For example, `ip`, `pod_uid`, `k8s.pod.ip`		no

exclude block

The exclude block configures which pods to exclude from the processor.

Note
Pods with the name jaeger-agent or jaeger-collector are excluded by default.

pod block

The pod block configures a pod to be excluded from the processor.

The following attributes are supported:

Name	Type	Description	Default	Required
`name`	`string`	The name of the pod		yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.k8sattributes is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.k8sattributes does not expose any component-specific debug information.

Examples

Basic usage

In most cases, this is enough to get started. It’ll add these resource attributes to all logs, metrics, and traces:

k8s.namespace.name
k8s.pod.name
k8s.pod.uid
k8s.pod.start_time
k8s.deployment.name
k8s.node.name

Example:

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = env("OTLP_ENDPOINT")
  }
}

Add additional metadata and labels

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
    logs    = [otelcol.processor.k8sattributes.default.input]
    traces  = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from      = "pod"
      key_regex = "(.*)/(.*)"
      tag_name  = "$1.$2"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.deployment.name",
      "k8s.statefulset.name",
      "k8s.daemonset.name",
      "k8s.cronjob.name",
      "k8s.job.name",
      "k8s.node.name",
      "k8s.pod.name",
      "k8s.pod.uid",
      "k8s.pod.start_time",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = env("OTLP_ENDPOINT")
  }
}

Adding Kubernetes metadata to Prometheus metrics

otelcol.processor.k8sattributes adds metadata to metrics signals in the form of resource attributes. To display the metadata as labels of Prometheus metrics, the OTLP attributes must be converted from resource attributes to datapoint attributes. One way to do this is by using an otelcol.processor.transform component.

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.k8sattributes.default.input]
  }
}

otelcol.processor.k8sattributes "default" {
  extract {
    label {
      from = "pod"
    }

    metadata = [
      "k8s.namespace.name",
      "k8s.pod.name",
    ]
  }

  output {
    metrics = [otelcol.processor.transform.add_kube_attrs.input]
  }
}

otelcol.processor.transform "add_kube_attrs" {
  error_mode = "ignore"

  metric_statements {
    context = "datapoint"
    statements = [
      "set(attributes[\"k8s.pod.name\"], resource.attributes[\"k8s.pod.name\"])",
      "set(attributes[\"k8s.namespace.name\"], resource.attributes[\"k8s.namespace.name\"])",
    ]
  }

  output {
    metrics = [otelcol.exporter.prometheus.default.input]
  }
}

otelcol.exporter.prometheus "default" {
  forward_to = [prometheus.remote_write.mimir.receiver]
}

prometheus.remote_write "mimir" {
  endpoint {
    url = "http://mimir:9009/api/v1/push"
  }
}

Compatible components

otelcol.processor.k8sattributes can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.k8sattributes has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.memory_limiter

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.memory_limiter

otelcol.processor.memory_limiter is used to prevent out of memory situations on a telemetry pipeline by performing periodic checks of memory usage. If usage exceeds the defined limits, data is dropped and garbage collections are triggered to reduce it.

The memory_limiter component uses both soft and hard limits, where the hard limit is always equal or larger than the soft limit. When memory usage goes above the soft limit, the processor component drops data and returns errors to the preceding components in the pipeline. When usage exceeds the hard limit, the processor forces a garbage collection in order to try and free memory. When usage is below the soft limit, no data is dropped and no forced garbage collection is performed.

NOTE: otelcol.processor.memory_limiter is a wrapper over the upstream OpenTelemetry Collector memorylimiter processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.processor.memory_limiter components can be specified by giving them different labels.

Usage

otelcol.processor.memory_limiter "LABEL" {
  check_interval = "1s"
  
  limit = "50MiB" // alternatively, set `limit_percentage` and `spike_limit_percentage`

  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.memory_limiter supports the following arguments:

Name	Type	Description	Default	Required
`check_interval`	`duration`	How often to check memory usage.		yes
`limit`	`string`	Maximum amount of memory targeted to be allocated by the process heap.	`"0MiB"`	no
`spike_limit`	`string`	Maximum spike expected between the measurements of memory usage.	20% of `limit`	no
`limit_percentage`	`int`	Maximum amount of total available memory targeted to be allocated by the process heap.	`0`	no
`spike_limit_percentage`	`int`	Maximum spike expected between the measurements of memory usage.	`0`	no

The arguments must define either limit or the limit_percentage, spike_limit_percentage pair, but not both.

The configuration options limit and limit_percentage define the hard limits. The soft limits are then calculated as the hard limit minus the spike_limit or spike_limit_percentage values respectively. The recommended value for spike limits is about 20% of the corresponding hard limit.

The recommended check_interval value is 1 second. If the traffic through the component is spiky in nature, it is recommended to either decrease the interval or increase the spike limit to avoid going over the hard limit.

The limit and spike_limit values must be larger than 1 MiB.

Blocks

The following blocks are supported inside the definition of otelcol.processor.memory_limiter:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.memory_limiter is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.memory_limiter does not expose any component-specific debug information.

Compatible components

otelcol.processor.memory_limiter can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.memory_limiter has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.probabilistic_sampler

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.probabilistic_sampler

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.processor.probabilistic_sampler accepts logs and traces data from other otelcol components and applies probabilistic sampling based on configuration options.

Note
otelcol.processor.probabilistic_sampler is a wrapper over the upstream OpenTelemetry Collector Contrib probabilistic_sampler processor. If necessary, bug reports or feature requests will be redirected to the upstream repository.

You can specify multiple otelcol.processor.probabilistic_sampler components by giving them different labels.

Usage

otelcol.processor.probabilistic_sampler "LABEL" {
  output {
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.probabilistic_sampler supports the following arguments:

Name	Type	Description	Default	Required
`hash_seed`	`uint32`	An integer used to compute the hash algorithm.	`0`	no
`sampling_percentage`	`float32`	Percentage of traces or logs sampled.	`0`	no
`attribute_source`	`string`	Defines where to look for the attribute in `from_attribute`.	`"traceID"`	no
`from_attribute`	`string`	The name of a log record attribute used for sampling purposes.	`""`	no
`sampling_priority`	`string`	The name of a log record attribute used to set a different sampling priority from the `sampling_percentage` setting.	`""`	no

hash_seed determines an integer to compute the hash algorithm. This argument could be used for both traces and logs. When used for logs, it computes the hash of a log record. For hashing to work, all collectors for a given tier, for example, behind the same load balancer, must have the same hash_seed. It is also possible to leverage a different hash_seed at different collector tiers to support additional sampling requirements.

sampling_percentage determines the percentage at which traces or logs are sampled. All traces or logs are sampled if you set this argument to a value greater than or equal to 100.

attribute_source (logs only) determines where to look for the attribute in from_attribute. The allowed values are traceID or record.

from_attribute (logs only) determines the name of a log record attribute used for sampling purposes, such as a unique log record ID. The value of the attribute is only used if the trace ID is absent or if attribute_source is set to record.

sampling_priority (logs only) determines the name of a log record attribute used to set a different sampling priority from the sampling_percentage setting. 0 means to never sample the log record, and greater than or equal to 100 means to always sample the log record.

The probabilistic_sampler supports two types of sampling for traces:

sampling.priority semantic convention as defined by OpenTracing.
Trace ID hashing.

The sampling.priority semantic convention takes priority over trace ID hashing. Trace ID hashing samples based on hash values determined by trace IDs.

The probabilistic_sampler supports sampling logs according to their trace ID, or by a specific log record attribute.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer OTLP-formatted data for any telemetry signal of these types:

logs
traces

Component health

otelcol.processor.probabilistic_sampler is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.probabilistic_sampler does not expose any component-specific debug information.

Examples

Basic usage

otelcol.processor.probabilistic_sampler "default" {
  hash_seed           = 123
  sampling_percentage = 15.3

  output {
    logs = [otelcol.exporter.otlp.default.input]
  }
}

Sample 15% of the logs

otelcol.processor.probabilistic_sampler "default" {
  sampling_percentage = 15

  output {
    logs = [otelcol.exporter.otlp.default.input]
  }
}

Sample logs according to their “logID” attribute

otelcol.processor.probabilistic_sampler "default" {
  sampling_percentage = 15
  attribute_source    = "record"
  from_attribute      = "logID"

  output {
    logs = [otelcol.exporter.otlp.default.input]
  }
}

Sample logs according to a “priority” attribute

otelcol.processor.probabilistic_sampler "default" {
  sampling_percentage = 15
  sampling_priority   = "priority"

  output {
    logs = [otelcol.exporter.otlp.default.input]
  }
}

Compatible components

otelcol.processor.probabilistic_sampler can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.probabilistic_sampler has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.resourcedetection

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.resourcedetection

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

otelcol.processor.resourcedetection detects resource information from the host in a format that conforms to the OpenTelemetry resource semantic conventions, and appends or overrides the resource values in the telemetry data with this information.

Note
otelcol.processor.resourcedetection is a wrapper over the upstream OpenTelemetry Collector Contrib resourcedetection processor. If necessary, bug reports or feature requests are redirected to the upstream repository.

You can specify multiple otelcol.processor.resourcedetection components by giving them different labels.

Usage

otelcol.processor.resourcedetection "LABEL" {
  output {
    logs    = [...]
    metrics = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.resourcedetection supports the following arguments:

Name	Type	Description	Default	Required
`detectors`	`list(string)`	An ordered list of named detectors used to detect resource information.	`["env"]`	no
`override`	`bool`	Configures whether existing resource attributes should be overridden or preserved.	`true`	no
`timeout`	`duration`	Timeout by which all specified detectors must complete.	`"5s"`	no

detectors could contain the following values:

env
ec2
ecs
eks
elasticbeanstalk
lambda
azure
aks
consul
docker
gcp
heroku
system
openshift
kubernetes_node

env is the only detector that is not configured through a River block. The env detector reads resource information from the OTEL_RESOURCE_ATTRIBUTES environment variable. This variable must be in the format <key1>=<value1>,<key2>=<value2>,..., the details of which are currently pending confirmation in the OpenTelemetry specification.

If a detector other than env is needed, you can customize it with the relevant River block. For example, you can customize the ec2 detector with the ec2 block. If you omit the ec2 block, the defaults specified in the ec2 block documentation are used.

If multiple detectors are inserting the same attribute name, the first detector to insert wins. For example, if you had detectors = ["eks", "ec2"] then cloud.platform will be aws_eks instead of ec2.

The following order is recommended for AWS:

Blocks

The following blocks are supported inside the definition of otelcol.processor.resourcedetection:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes
ec2	ec2		no
ecs	ecs		no
eks	eks		no
elasticbeanstalk	elasticbeanstalk		no
lambda	lambda		no
azure	azure		no
aks	aks		no
consul	consul		no
docker	docker		no
gcp	gcp		no
heroku	heroku		no
system	system		no
openshift	openshift		no
kubernetes_node	kubernetes_node		no

output

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

ec2

The ec2 block reads resource information from the EC2 instance metadata API using the AWS SDK for Go.

The ec2 block supports the following attributes:

Attribute	Type	Description	Default	Required
`tags`	`list(string)`	A list of regular expressions to match against tag keys of an EC2 instance.	`[]`	no

If you are using a proxy server on your EC2 instance, it’s important that you exempt requests for instance metadata as described in the AWS cli user guide. Failing to do so can result in proxied or missing instance data.

If the instance is part of AWS ParallelCluster and the detector is failing to connect to the metadata server, check the iptable and make sure the chain PARALLELCLUSTER_IMDS contains a rule that allows the Grafana Agent user to access 169.254.169.254/32.

tags can be used to gather tags for the EC2 instance which Grafana Agent is running on. To fetch EC2 tags, the IAM role assigned to the EC2 instance must have a policy that includes the ec2:DescribeTags permission.

The ec2 block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

ec2 > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.account.id	Toggles the `cloud.account.id` resource attribute. Sets `enabled` to `true` by default.	no
cloud.availability_zone	Toggles the `cloud.availability_zone` resource attribute. Sets `enabled` to `true` by default.	no
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
host.id	Toggles the `host.id` resource attribute. Sets `enabled` to `true` by default.	no
host.image.id	Toggles the `host.image.id` resource attribute. Sets `enabled` to `true` by default.	no
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no
host.type	Toggles the `host.type` resource attribute. Sets `enabled` to `true` by default.	no

ecs

The ecs block queries the Task Metadata Endpoint (TMDE) to record information about the current ECS Task. Only TMDE V4 and V3 are supported.

The ecs block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

ecs > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
aws.ecs.cluster.arn	Toggles the `aws.ecs.cluster.arn` resource attribute. Sets `enabled` to `true` by default.	no
aws.ecs.launchtype	Toggles the `aws.ecs.launchtype` resource attribute. Sets `enabled` to `true` by default.	no
aws.ecs.task.arn	Toggles the `aws.ecs.task.arn` resource attribute. Sets `enabled` to `true` by default.	no
aws.ecs.task.family	Toggles the `aws.ecs.task.family` resource attribute. Sets `enabled` to `true` by default.	no
aws.ecs.task.id	Toggles the `aws.ecs.task.id` resource attribute. Sets `enabled` to `true` by default.	no
aws.ecs.task.revision	Toggles the `aws.ecs.task.revision` resource attribute. Sets `enabled` to `true` by default.	no
aws.log.group.arns	Toggles the `aws.log.group.arns` resource attribute. Sets `enabled` to `true` by default.	no
aws.log.group.names	Toggles the `aws.log.group.names` resource attribute. Sets `enabled` to `true` by default.	no
aws.log.stream.arns	Toggles the `aws.log.stream.arns` resource attribute. Sets `enabled` to `true` by default.	no
aws.log.stream.names	Toggles the `aws.log.stream.names` resource attribute. Sets `enabled` to `true` by default.	no
cloud.account.id	Toggles the `cloud.account.id` resource attribute. Sets `enabled` to `true` by default.	no
cloud.availability_zone	Toggles the `cloud.availability_zone` resource attribute. Sets `enabled` to `true` by default.	no
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no

eks

The eks block adds resource attributes for Amazon EKS.

The eks block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

eks > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
k8s.cluster.name	Toggles the `k8s.cluster.name` resource attribute. Sets `enabled` to `false` by default.	no

Example values:

cloud.provider: "aws"
cloud.platform: "aws_eks"

elasticbeanstalk

The elasticbeanstalk block reads the AWS X-Ray configuration file available on all Beanstalk instances with X-Ray Enabled.

The elasticbeanstalk block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

elasticbeanstalk > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
deployment.envir	Toggles the `deployment.envir` resource attribute. Sets `enabled` to `true` by default.	no
service.instance	Toggles the `service.instance` resource attribute. Sets `enabled` to `true` by default.	no
service.version	Toggles the `service.version` resource attribute. Sets `enabled` to `true` by default.	no

Example values:

cloud.provider: "aws"
cloud.platform: "aws_elastic_beanstalk"

lambda

The lambda block uses the AWS Lambda runtime environment variables to retrieve various resource attributes.

The lambda block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

lambda > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
aws.log.group.names	Toggles the `aws.log.group.names` resource attribute. Sets `enabled` to `true` by default.	no
aws.log.stream.names	Toggles the `aws.log.stream.names` resource attribute. Sets `enabled` to `true` by default.	no
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
faas.instance	Toggles the `faas.instance` resource attribute. Sets `enabled` to `true` by default.	no
faas.max_memory	Toggles the `faas.max_memory` resource attribute. Sets `enabled` to `true` by default.	no
faas.name	Toggles the `faas.name` resource attribute. Sets `enabled` to `true` by default.	no
faas.version	Toggles the `faas.version` resource attribute. Sets `enabled` to `true` by default.	no

Cloud semantic conventions:

cloud.provider: "aws"
cloud.platform: "aws_lambda"
cloud.region: $AWS_REGION

Function as a Service semantic conventions and AWS Lambda semantic conventions:

faas.name: $AWS_LAMBDA_FUNCTION_NAME
faas.version: $AWS_LAMBDA_FUNCTION_VERSION
faas.instance: $AWS_LAMBDA_LOG_STREAM_NAME
faas.max_memory: $AWS_LAMBDA_FUNCTION_MEMORY_SIZE

AWS Logs semantic conventions:

aws.log.group.names: $AWS_LAMBDA_LOG_GROUP_NAME
aws.log.stream.names: $AWS_LAMBDA_LOG_STREAM_NAME

azure

The azure block queries the Azure Instance Metadata Service to retrieve various resource attributes.

The azure block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

azure > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
azure.resourcegroup.name	Toggles the `azure.resourcegroup.name` resource attribute. Sets `enabled` to `true` by default.	no
azure.vm.name	Toggles the `azure.vm.name` resource attribute. Sets `enabled` to `true` by default.	no
azure.vm.scaleset.name	Toggles the `azure.vm.scaleset.name` resource attribute. Sets `enabled` to `true` by default.	no
azure.vm.size	Toggles the `azure.vm.size` resource attribute. Sets `enabled` to `true` by default.	no
cloud.account.id	Toggles the `cloud.account.id` resource attribute. Sets `enabled` to `true` by default.	no
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
host.id	Toggles the `host.id` resource attribute. Sets `enabled` to `true` by default.	no
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no

Example values:

cloud.provider: "azure"
cloud.platform: "azure_vm"

aks

The aks block adds resource attributes related to Azure AKS.

The aks block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

aks > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
k8s.cluster.name	Toggles the `k8s.cluster.name` resource attribute. Sets `enabled` to `false` by default.	no

Example values:

cloud.provider: "azure"
cloud.platform: "azure_vm"

Azure AKS cluster name is derived from the Azure Instance Metadata Service’s (IMDS) infrastructure resource group field. This field contains the resource group and name of the cluster, separated by underscores. For example: MC_<resource group>_<cluster name>_<location>.

Example:

Resource group: my-resource-group
Cluster name: my-cluster
Location: eastus
Generated name: MC_my-resource-group_my-cluster_eastus

The cluster name is detected if it does not contain underscores and if a custom infrastructure resource group name was not used.

If accurate parsing cannot be performed, the infrastructure resource group value is returned. This value can be used to uniquely identify the cluster, because Azure will not allow users to create multiple clusters with the same infrastructure resource group name.

consul

The consul block queries a Consul agent and reads its configuration endpoint to retrieve values for resource attributes.

The consul block supports the following attributes:

Attribute	Type	Description	Default	Required
`address`	`string`	The address of the Consul server	`""`	no
`datacenter`	`string`	Datacenter to use. If not provided, the default agent datacenter is used.	`""`	no
`token`	`secret`	A per-request ACL token which overrides the Consul agent’s default (empty) token.	`""`	no
`namespace`	`string`	The name of the namespace to send along for the request.	`""`	no
`meta`	`list(string)`	Allowlist of Consul Metadata keys to use as resource attributes.	`[]`	no

token is only required if Consul’s ACL System is enabled.

The consul block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

consul > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
host.id	Toggles the `host.id` resource attribute. Sets `enabled` to `true` by default.	no
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no

docker

The docker block queries the Docker daemon to retrieve various resource attributes from the host machine.

You need to mount the Docker socket (/var/run/docker.sock on Linux) to contact the Docker daemon. Docker detection does not work on MacOS.

The docker block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

docker > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no
os.type	Toggles the `os.type` resource attribute. Sets `enabled` to `true` by default.	no

gcp

The gcp block detects resource attributes using the Google Cloud Client Libraries for Go, which reads resource information from the GCP metadata server. The detector also uses environment variables to identify which GCP platform the application is running on, and assigns appropriate resource attributes for that platform.

Use the gcp detector regardless of the GCP platform Grafana Agent is running on.

The gcp block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

gcp > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.account.id	Toggles the `cloud.account.id` resource attribute. Sets `enabled` to `true` by default.	no
cloud.availability_zone	Toggles the `cloud.availability_zone` resource attribute. Sets `enabled` to `true` by default.	no
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
faas.id	Toggles the `faas.id` resource attribute. Sets `enabled` to `true` by default.	no
faas.instance	Toggles the `faas.instance` resource attribute. Sets `enabled` to `true` by default.	no
faas.name	Toggles the `faas.name` resource attribute. Sets `enabled` to `true` by default.	no
faas.version	Toggles the `faas.version` resource attribute. Sets `enabled` to `true` by default.	no
gcp.cloud_run.job.execution	Toggles the `gcp.cloud_run.job.execution` resource attribute. Sets `enabled` to `true` by default.	no
gcp.cloud_run.job.task_index	Toggles the `gcp.cloud_run.job.task_index` resource attribute. Sets `enabled` to `true` by default.	no
gcp.gce.instance.hostname	Toggles the `gcp.gce.instance.hostname` resource attribute. Sets `enabled` to `false` by default.	no
gcp.gce.instance.name	Toggles the `gcp.gce.instance.name` resource attribute. Sets `enabled` to `false` by default.	no
host.id	Toggles the `host.id` resource attribute. Sets `enabled` to `true` by default.	no
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no
host.type	Toggles the `host.type` resource attribute. Sets `enabled` to `true` by default.	no
k8s.cluster.name	Toggles the `k8s.cluster.name` resource attribute. Sets `enabled` to `true` by default.	no

Google Compute Engine (GCE) metadata

cloud.provider: "gcp"
cloud.platform: "gcp_compute_engine"
cloud.account.id: project id
cloud.region: e.g. "us-central1"
cloud.availability_zone: e.g. "us-central1-c"
host.id: instance id
host.name: instance name
host.type: machine type
(optional) gcp.gce.instance.hostname
(optional) gcp.gce.instance.name

Google Kubernetes Engine (GKE) metadata

cloud.provider: "gcp"
cloud.platform: "gcp_kubernetes_engine"
cloud.account.id: project id
cloud.region: only for regional GKE clusters; e.g. "us-central1"
cloud.availability_zone: only for zonal GKE clusters; e.g. "us-central1-c"
k8s.cluster.name
host.id: instance id
host.name: instance name; only when workload identity is disabled

One known issue happens when GKE workload identity is enabled. The GCE metadata endpoints won’t be available, and the GKE resource detector won’t be able to determine host.name. If this happens, you can set host.name from one of the following resources:

Get the node.name through the downward API with the env detector.
Get the Kubernetes node name from the Kubernetes API (with k8s.io/client-go).

Google Cloud Run Services metadata

cloud.provider: "gcp"
cloud.platform: "gcp_cloud_run"
cloud.account.id: project id
cloud.region: e.g. "us-central1"
faas.id: instance id
faas.name: service name
faas.version: service revision

Cloud Run Jobs metadata

cloud.provider: "gcp"
cloud.platform: "gcp_cloud_run"
cloud.account.id: project id
cloud.region: e.g. "us-central1"
faas.id: instance id
faas.name: service name
gcp.cloud_run.job.execution: e.g. "my-service-ajg89"
gcp.cloud_run.job.task_index: e.g. "0"

Google Cloud Functions metadata

cloud.provider: "gcp"
cloud.platform: "gcp_cloud_functions"
cloud.account.id: project id
cloud.region: e.g. "us-central1"
faas.id: instance id
faas.name: function name
faas.version: function version

Google App Engine metadata

cloud.provider: "gcp"
cloud.platform: "gcp_app_engine"
cloud.account.id: project id
cloud.region: e.g. "us-central1"
cloud.availability_zone: e.g. "us-central1-c"
faas.id: instance id
faas.name: service name
faas.version: service version

heroku

The heroku block adds resource attributes derived from Heroku dyno metadata.

The heroku block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

heroku > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
heroku.app.id	Toggles the `heroku.app.id` resource attribute. Sets `enabled` to `true` by default.	no
heroku.dyno.id	Toggles the `heroku.dyno.id` resource attribute. Sets `enabled` to `true` by default.	no
heroku.release.commit	Toggles the `heroku.release.commit` resource attribute. Sets `enabled` to `true` by default.	no
heroku.release.creation_timestamp	Toggles the `heroku.release.creation_timestamp` resource attribute. Sets `enabled` to `true` by default.	no
service.instance.id	Toggles the `service.instance.id` resource attribute. Sets `enabled` to `true` by default.	no
service.name	Toggles the `service.name` resource attribute. Sets `enabled` to `true` by default.	no
service.version	Toggles the `service.version` resource attribute. Sets `enabled` to `true` by default.	no

When Heroku dyno metadata is active, Heroku applications publish information through environment variables. We map these environment variables to resource attributes as follows:

Dyno metadata environment variable	Resource attribute
`HEROKU_APP_ID`	`heroku.app.id`
`HEROKU_APP_NAME`	`service.name`
`HEROKU_DYNO_ID`	`service.instance.id`
`HEROKU_RELEASE_CREATED_AT`	`heroku.release.creation_timestamp`
`HEROKU_RELEASE_VERSION`	`service.version`
`HEROKU_SLUG_COMMIT`	`heroku.release.commit`

For more information, see the Heroku cloud provider documentation under the OpenTelemetry specification semantic conventions.

system

The system block queries the host machine to retrieve various resource attributes.

Note
Use the Docker detector if running Grafana Agent as a Docker container.

The system block supports the following attributes:

Attribute	Type	Description	Default	Required
`hostname_sources`	`list(string)`	A priority list of sources from which the hostname will be fetched.	`["dns", "os"]`	no

The valid options for hostname_sources are:

"dns": Uses multiple sources to get the fully qualified domain name. Firstly, it looks up the host name in the local machine’s hosts file. If that fails, it looks up the CNAME. Lastly, if that fails, it does a reverse DNS query. Note: this hostname source may produce unreliable results on Windows. To produce a FQDN, Windows hosts might have better results using the “lookup” hostname source, which is mentioned below.
"os": Provides the hostname provided by the local machine’s kernel.
"cname": Provides the canonical name, as provided by net.LookupCNAME in the Go standard library. Note: this hostname source may produce unreliable results on Windows.
"lookup": Does a reverse DNS lookup of the current host’s IP address.

In case of an error in fetching a hostname from a source, the next source from the list of hostname_sources will be considered.

The system block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

system > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
host.arch	Toggles the `host.arch` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.cache.l2.size	Toggles the `host.cpu.cache.l2.size` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.family	Toggles the `host.cpu.family` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.model.id	Toggles the `host.cpu.model.id` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.model.name	Toggles the `host.cpu.model.name` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.stepping	Toggles the `host.cpu.stepping` resource attribute. Sets `enabled` to `false` by default.	no
host.cpu.vendor.id	Toggles the `host.cpu.vendor.id` resource attribute. Sets `enabled` to `false` by default.	no
host.id	Toggles the `host.id` resource attribute. Sets `enabled` to `false` by default.	no
host.ip	Toggles the `host.ip` resource attribute. Sets `enabled` to `false` by default.	no
host.mac	Toggles the `host.mac` resource attribute. Sets `enabled` to `false` by default.	no
host.name	Toggles the `host.name` resource attribute. Sets `enabled` to `true` by default.	no
os.description	Toggles the `os.description` resource attribute. Sets `enabled` to `false` by default.	no
os.type	Toggles the `os.type` resource attribute. Sets `enabled` to `true` by default.	no

openshift

The openshift block queries the OpenShift and Kubernetes APIs to retrieve various resource attributes.

The openshift block supports the following attributes:

Attribute	Type	Description	Default	Required
`address`	`string`	Address of the OpenShift API server.	See below	no
`token`	`string`	Token used to identify against the OpenShift API server.	""	no

The “get”, “watch”, and “list” permissions are required:

kind: ClusterRole
metadata:
  name: grafana-agent
rules:
- apiGroups: ["config.openshift.io"]
  resources: ["infrastructures", "infrastructures/status"]
  verbs: ["get", "watch", "list"]

By default, the API address is determined from the environment variables KUBERNETES_SERVICE_HOST, KUBERNETES_SERVICE_PORT and the service token is read from /var/run/secrets/kubernetes.io/serviceaccount/token. If TLS is not explicitly disabled and no ca_file is configured, /var/run/secrets/kubernetes.io/serviceaccount/ca.crt is used. The determination of the API address, ca_file, and the service token is skipped if they are set in the configuration.

The openshift block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no
tls	TLS settings for the connection with the OpenShift API.	yes

openshift > tls

The tls block configures TLS settings used for the connection to the gRPC server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

openshift > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
cloud.platform	Toggles the `cloud.platform` resource attribute. Sets `enabled` to `true` by default.	no
cloud.provider	Toggles the `cloud.provider` resource attribute. Sets `enabled` to `true` by default.	no
cloud.region	Toggles the `cloud.region` resource attribute. Sets `enabled` to `true` by default.	no
k8s.cluster.name	Toggles the `k8s.cluster.name` resource attribute. Sets `enabled` to `true` by default.	no

kubernetes_node

The kubernetes_node block queries the Kubernetes API server to retrieve various node resource attributes.

The kubernetes_node block supports the following attributes:

Attribute	Type	Description	Default	Required
`auth_type`	`string`	Configures how to authenticate to the K8s API server.	`"none"`	no
`context`	`string`	Override the current context when `auth_type` is set to `"kubeConfig"`.	`""`	no
`node_from_env_var`	`string`	The name of an environment variable from which to retrieve the node name.	`"K8S_NODE_NAME"`	no

The “get” and “list” permissions are required:

kind: ClusterRole
metadata:
  name: grafana-agent
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list"]

auth_type can be set to one of the following:

none: no authentication.
serviceAccount: use the standard service account token provided to the Grafana Agent pod.
kubeConfig: use credentials from ~/.kube/config.

The kubernetes_node block supports the following blocks:

Block	Description	Required
resource_attributes	Configures which resource attributes to add.	no

kubernetes_node > resource_attributes

The resource_attributes block supports the following blocks:

Block	Description	Required
k8s.node.name	Toggles the `k8s.node.name` resource attribute. Sets `enabled` to `true` by default.	no
k8s.node.uid	Toggles the `k8s.node.uid` resource attribute. Sets `enabled` to `true` by default.	no

Common configuration

Resource attribute config

This block describes how to configure resource attributes such as k8s.node.name and azure.vm.name. Every block is configured using the same set of attributes. Only the default values for those attributes might differ across resource attributes. For example, some resource attributes have enabled set to true by default, whereas others don’t.

The following attributes are supported:

Attribute	Type	Description	Default	Required
`enabled`	`bool`	Toggles whether to add the resource attribute to the span, log, or metric resource.	See below	no

To see the default value for enabled, refer to the tables in the sections above which list the resource attributes blocks. The “Description” column will state either…

Sets enabled to true by default.

… or:

Sets enabled to false by default.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer OTLP-formatted data for any telemetry signal of these types:

logs
metrics
traces

Component health

otelcol.processor.resourcedetection is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.resourcedetection doesn’t expose any component-specific debug information.

Examples

env detector

If you set up a OTEL_RESOURCE_ATTRIBUTES environment variable with value of TestKey=TestValue, then all logs, metrics, and traces have a resource attribute with a key TestKey and value of TestValue.

otelcol.processor.resourcedetection "default" {
  detectors = ["env"]

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

env and ec2

There is no need to put in an ec2 {} River block. The ec2 defaults are applied automatically, as specified in ec2.

otelcol.processor.resourcedetection "default" {
  detectors = ["env", "ec2"]

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

ec2 with default resource attributes

There is no need to put in a ec2 {} River block. The ec2 defaults are applied automatically, as specified in ec2.

otelcol.processor.resourcedetection "default" {
  detectors = ["ec2"]

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

ec2 with explicit resource attributes

otelcol.processor.resourcedetection "default" {
  detectors = ["ec2"]
  ec2 {
    tags = ["^tag1$", "^tag2$", "^label.*$"]
    resource_attributes {
      cloud.account.id  { enabled = true }
      cloud.availability_zone  { enabled = true }
      cloud.platform  { enabled = true }
      cloud.provider  { enabled = true }
      cloud.region  { enabled = true }
      host.id  { enabled = true }
      host.image.id  { enabled = false }
      host.name  { enabled = false }
      host.type  { enabled = false }
    }
  }

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

kubernetes_node

This example uses the default node_from_env_var option of K8S_NODE_NAME.

There is no need to put in a kubernetes_node {} River block. The kubernetes_node defaults are applied automatically, as specified in kubernetes_node.

otelcol.processor.resourcedetection "default" {
  detectors = ["kubernetes_node"]

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

You need to add this to your workload:

        env:
          - name: K8S_NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName

kubernetes_node with a custom environment variable

This example uses a custom node_from_env_var set to my_custom_var.

otelcol.processor.resourcedetection "default" {
  detectors = ["kubernetes_node"]
  kubernetes_node {
    node_from_env_var = "my_custom_var"
  }

  output {
    logs    = [otelcol.exporter.otlp.default.input]
    metrics = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

You need to add this to your workload:

        env:
          - name: my_custom_var
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName

Compatible components

otelcol.processor.resourcedetection can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.resourcedetection has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.span

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.span

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.processor.span accepts traces telemetry data from other otelcol components and modifies the names and attributes of the spans. It also supports the ability to filter input data to determine if it should be included or excluded from this processor.

NOTE: otelcol.processor.span is a wrapper over the upstream OpenTelemetry Collector span processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

You can specify multiple otelcol.processor.span components by giving them different labels.

Usage

otelcol.processor.span "LABEL" {
  output {
    traces  = [...]
  }
}

Arguments

otelcol.processor.span doesn’t support any arguments and is configured fully through inner blocks.

Blocks

The following blocks are supported inside the definition of otelcol.processor.span:

Hierarchy	Block	Description	Required
output	output	Configures where to send received telemetry data.	yes
name	name	Configures how to rename a span and add attributes.	no
name > to_attributes	to-attributes	Configuration to create attributes from a span name.	no
status	status	Specifies a status which should be set for this span.	no
include	include	Filter for data included in this processor’s actions.	no
include > regexp	regexp	Regex cache settings.	no
include > attribute	attribute	A list of attributes to match against.	no
include > resource	resource	A list of items to match the resources against.	no
include > library	library	A list of items to match the implementation library against.	no
exclude	exclude	Filter for data excluded from this processor’s actions	no
exclude > regexp	regexp	Regex cache settings.	no
exclude > attribute	attribute	A list of attributes to match against.	no
exclude > resource	resource	A list of items to match the resources against.	no
exclude > library	library	A list of items to match the implementation library against.	no

The > symbol indicates deeper levels of nesting. For example, include > attribute refers to an attribute block defined inside an include block.

If both an include block and an excludeblock are specified, the include properties are checked before the exclude properties.

name block

The name block configures how to rename a span and add attributes.

The following attributes are supported:

Name	Type	Description	Default	Required
`from_attributes`	`list(string)`	Attribute keys to pull values from, to generate a new span name.	`[]`	no
`separator`	`string`	Separates attributes values in the new span name.	`""`	no

Firstly from_attributes rules are applied, then to-attributes are applied. At least one of these 2 fields must be set.

from_attributes represents the attribute keys to pull the values from to generate the new span name:

All attribute keys are required in the span to rename a span. If any attribute is missing from the span, no rename will occur.
The new span name is constructed in order of the from_attributes specified in the configuration.

separator is the string used to separate attributes values in the new span name. If no value is set, no separator is used between attribute values. separator is used with from_attributes only; it is not used with to-attributes.

to_attributes block

The to_attributes block configures how to create attributes from a span name.

The following attributes are supported:

Name	Type	Description	Default	Required
`rules`	`list(string)`	A list of regex rules to extract attribute values from span name.		yes
`break_after_match`	`bool`	Configures if processing of rules should stop after the first match.	`false`	no

Each rule in the rules list is a regex pattern string.

The span name is checked against each regex in the list.
If it matches, then all named subexpressions of the regex are extracted as attributes and are added to the span.
Each subexpression name becomes an attribute name and the subexpression matched portion becomes the attribute value.
The matched portion in the span name is replaced by extracted attribute name.
If the attributes already exist in the span then they will be overwritten.
The process is repeated for all rules in the order they are specified.
Each subsequent rule works on the span name that is the output after processing the previous rule.

break_after_match specifies if processing of rules should stop after the first match. If it is false, rule processing will continue to be performed over the modified span name.

status block

The status block specifies a status which should be set for this span.

The following attributes are supported:

Name	Type	Description	Default	Required
`code`	`string`	A status code.		yes
`description`	`string`	An optional field documenting Error status codes.	`""`	no

The supported values for code are:

Ok
Error
Unset

description should only be specified if code is set to Error.

include block

The include block provides an option to include data being fed into the name and status blocks based on the properties of a span.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_type`	`string`	Controls how items to match against are interpreted.		yes
`services`	`list(string)`	A list of items to match the service name against.	`[]`	no
`span_names`	`list(string)`	A list of items to match the span name against.	`[]`	no
`span_kinds`	`list(string)`	A list of items to match the span kind against.	`[]`	no

match_type is required and must be set to either "regexp" or "strict".

A match occurs if at least one item in the lists matches.

One of services, span_names, span_kinds, attribute, resource, or library must be specified with a non-empty value for a valid configuration.

exclude block

The exclude block provides an option to exclude data from being fed into the name and status blocks based on the properties of a span.

The following arguments are supported:

Name	Type	Description	Default	Required
`match_type`	`string`	Controls how items to match against are interpreted.		yes
`services`	`list(string)`	A list of items to match the service name against.	`[]`	no
`span_names`	`list(string)`	A list of items to match the span name against.	`[]`	no
`span_kinds`	`list(string)`	A list of items to match the span kind against.	`[]`	no

match_type is required and must be set to either "regexp" or "strict".

A match occurs if at least one item in the lists matches.

One of services, span_names, span_kinds, attribute, resource, or library must be specified with a non-empty value for a valid configuration.

regexp block

This block is an optional configuration for the match_type of "regexp". It configures a Least Recently Used (LRU) cache.

The following arguments are supported:

Name	Type	Description	Default	Required
`cache_enabled`	`bool`	Determines whether match results are LRU cached.	`false`	no
`cache_max_num_entries`	`int`	The max number of entries of the LRU cache that stores match results.	`0`	no

Enabling cache_enabled could make subsequent matches faster. Cache size is unlimited unless cache_max_num_entries is also specified.

cache_max_num_entries is ignored if cache_enabled is false.

attribute block

This block specifies an attribute to match against:

More than one attribute block can be defined.
Only match_type = "strict" is allowed if attribute is specified.
All attribute blocks must match exactly for a match to occur.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	The attribute key.		yes
`value`	`any`	The attribute value to match against.		no

If value isn’t set, any value will match. The type of value could be a number, a string, or a boolean.

resource block

This block specifies items to match the resources against:

More than one resource block can be defined.
A match occurs if the input data resources match at least one resource block.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	The resource key.		yes
`value`	`any`	The resource value to match against.		no

If value is not set, any value will match. The type of value could be a number, a string, or a boolean.

library block

This block specifies properties to match the implementation library against:

More than one library block can be defined.
A match occurs if the span’s implementation library matches at least one library block.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The attribute key.		yes
`version`	`string`	The version to match against.	null	no

If version is unset, any version matches. If version is set to an empty string, it only matches a library version, which is also an empty string.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the traces argument accordingly to send telemetry data to other components.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer OTLP-formatted data for traces telemetry signals. Logs and metrics are not supported.

Component health

otelcol.processor.attributes is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.attributes does not expose any component-specific debug information.

Examples

Creating a new span name from attribute values

This example creates a new span name from the values of attributes db.svc, operation, and id, in that order, separated by the value ::. All attribute keys need to be specified in the span for the processor to rename it.

otelcol.processor.span "default" {
  name {
    separator        = "::"
    from_attributes  = ["db.svc", "operation", "id"]
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

For a span with the following attributes key/value pairs, the above Flow configuration will change the span name to "location::get::1234":

{ 
  "db.svc": "location", 
  "operation": "get", 
  "id": "1234"
}

For a span with the following attributes key/value pairs, the above Flow configuration will not change the span name. This is because the attribute key operation isn’t set:

{ 
  "db.svc": "location", 
  "id": "1234"
}

Creating a new span name from attribute values (no separator)

otelcol.processor.span "default" {
  name {
    from_attributes = ["db.svc", "operation", "id"]
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

For a span with the following attributes key/value pairs, the above Flow configuration will change the span name to "locationget1234":

{ 
  "db.svc": "location", 
  "operation": "get", 
  "id": "1234"
}

Renaming a span name and adding attributes

Example input and output using the Flow configuration below:

Let’s assume input span name is /api/v1/document/12345678/update
The span name will be changed to /api/v1/document/{documentId}/update
A new attribute "documentId"="12345678" will be added to the span.

otelcol.processor.span "default" {
  name {
    to_attributes {
      rules = ["^\\/api\\/v1\\/document\\/(?P<documentId>.*)\\/update$"]
    }
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

Filtering, renaming a span name and adding attributes

This example renames the span name to {operation_website} and adds the attribute {Key: operation_website, Value: <old span name> } if the span has the following properties:

Service name contains the word banks.
The span name contains / anywhere in the string.
The span name is not donot/change.

otelcol.processor.span "default" {
  include {
    match_type = "regexp"
    services   = ["banks"]
    span_names = ["^(.*?)/(.*?)$"]
  }
  exclude {
    match_type = "strict"
    span_names = ["donot/change"]
  }
  name {
    to_attributes {
      rules = ["(?P<operation_website>.*?)$"]
    }
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

Setting a status

This example changes the status of a span to “Error” and sets an error description.

otelcol.processor.span "default" {
  status {
    code        = "Error"
    description = "some additional error description"
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

Setting a status depending on an attribute value

This example sets the status to success only when attribute http.status_code is equal to 400.

otelcol.processor.span "default" {
  include {
    match_type = "strict"
    attribute {
      key   = "http.status_code"
      value = 400
    }
  }
  status {
    code = "Ok"
  }

  output {
      traces = [otelcol.exporter.otlp.default.input]
  }
}

Compatible components

otelcol.processor.span can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.span has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.tail_sampling

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.tail_sampling

BETA: This is a beta component. Beta components are subject to breaking changes, and may be replaced with equivalent functionality that cover the same use case.

otelcol.processor.tail_sampling samples traces based on a set of defined policies. All spans for a given trace must be received by the same collector instance for effective sampling decisions.

The tail_sampling component uses both soft and hard limits, where the hard limit is always equal or larger than the soft limit. When memory usage goes above the soft limit, the processor component drops data and returns errors to the preceding components in the pipeline. When usage exceeds the hard limit, the processor forces a garbage collection in order to try and free memory. When usage is below the soft limit, no data is dropped and no forced garbage collection is performed.

Note: otelcol.processor.tail_sampling is a wrapper over the upstream OpenTelemetry Collector Contrib tail_sampling processor. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.processor.tail_sampling components can be specified by giving them different labels.

Usage

otelcol.processor.tail_sampling "LABEL" {
  policy {
    ...
  }
  ...

  output {
    traces  = [...]
  }
}

Arguments

otelcol.processor.tail_sampling supports the following arguments:

Name	Type	Description	Default	Required
`decision_wait`	`duration`	Wait time since the first span of a trace before making a sampling decision.	`"30s"`	no
`num_traces`	`int`	Number of traces kept in memory.	`50000`	no
`expected_new_traces_per_sec`	`int`	Expected number of new traces (helps in allocating data structures).	`0`	no

decision_wait determines the number of batches to maintain on a channel. Its value must convert to a number of seconds greater than zero.

num_traces determines the buffer size of the trace delete channel which is composed of trace ids. Increasing the number will increase the memory usage of the component while decreasing the number will lower the maximum amount of traces kept in memory.

expected_new_traces_per_sec determines the initial slice sizing of the current batch. A larger number will use more memory but be more efficient when adding traces to the batch.

Blocks

The following blocks are supported inside the definition of otelcol.processor.tail_sampling:

Hierarchy	Block	Description	Required
policy	policy []	Policies used to make a sampling decision.	yes
policy > latency	latency	The policy will sample based on the duration of the trace.	no
policy > numeric_attribute	numeric_attribute	The policy will sample based on number attributes (resource and record).	no
policy > probabilistic	probabilistic	The policy will sample a percentage of traces.	no
policy > status_code	status_code	The policy will sample based upon the status code.	no
policy > string_attribute	string_attribute	The policy will sample based on string attributes (resource and record) value matches.	no
policy > rate_limiting	rate_limiting	The policy will sample based on rate.	no
policy > span_count	span_count	The policy will sample based on the minimum number of spans within a batch.	no
policy > boolean_attribute	boolean_attribute	The policy will sample based on a boolean attribute (resource and record).	no
policy > ottl_condition	ottl_condition	The policy will sample based on a given boolean OTTL condition (span and span event).	no
policy > trace_state	trace_state	The policy will sample based on TraceState value matches.	no
policy > and	and	The policy will sample based on multiple policies, creates an `and` policy.	no
policy > and > and_sub_policy	and_sub_policy []	A set of policies underneath an `and` policy type.	no
policy > and > and_sub_policy > latency	latency	The policy will sample based on the duration of the trace.	no
policy > and > and_sub_policy > numeric_attribute	numeric_attribute	The policy will sample based on number attributes (resource and record).	no
policy > and > and_sub_policy > probabilistic	probabilistic	The policy will sample a percentage of traces.	no
policy > and > and_sub_policy > status_code	status_code	The policy will sample based upon the status code.	no
policy > and > and_sub_policy > string_attribute	string_attribute	The policy will sample based on string attributes (resource and record) value matches.	no
policy > and > and_sub_policy > rate_limiting	rate_limiting	The policy will sample based on rate.	no
policy > and > and_sub_policy > span_count	span_count	The policy will sample based on the minimum number of spans within a batch.	no
policy > and > and_sub_policy > boolean_attribute	boolean_attribute	The policy will sample based on a boolean attribute (resource and record).	no
policy > and > and_sub_policy > ottl_condition	ottl_condition	The policy will sample based on a given boolean OTTL condition (span and span event).	no
policy > and > and_sub_policy > trace_state	trace_state	The policy will sample based on TraceState value matches.	no
policy > composite	composite	This policy will sample based on a combination of above samplers, with ordering and rate allocation per sampler.	no
policy > composite > composite_sub_policy	composite_sub_policy []	A set of policies underneath a `composite` policy type.	no
policy > composite > composite_sub_policy > latency	latency	The policy will sample based on the duration of the trace.	no
policy > composite > composite_sub_policy > numeric_attribute	numeric_attribute	The policy will sample based on number attributes (resource and record).	no
policy > composite > composite_sub_policy > probabilistic	probabilistic	The policy will sample a percentage of traces.	no
policy > composite > composite_sub_policy > status_code	status_code	The policy will sample based upon the status code.	no
policy > composite > composite_sub_policy > string_attribute	string_attribute	The policy will sample based on string attributes (resource and record) value matches.	no
policy > composite > composite_sub_policy > rate_limiting	rate_limiting	The policy will sample based on rate.	no
policy > composite > composite_sub_policy > span_count	span_count	The policy will sample based on the minimum number of spans within a batch.	no
policy > composite > composite_sub_policy > boolean_attribute	boolean_attribute	The policy will sample based on a boolean attribute (resource and record).	no
policy > composite > composite_sub_policy > ottl_condition	ottl_condition	The policy will sample based on a given boolean OTTL condition (span and span event).	no
policy > composite > composite_sub_policy > trace_state	trace_state	The policy will sample based on TraceState value matches.	no
output	output []	Configures where to send received telemetry data.	yes

policy block

The policy block configures a sampling policy used by the component. At least one policy block is required.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The custom name given to the policy.		yes
`type`	`string`	The valid policy type for this policy.		yes

Each policy results in a decision, and the processor evaluates them to make a final decision:

When there’s an “inverted not sample” decision, the trace is not sampled.
When there’s a “sample” decision, the trace is sampled.
When there’s an “inverted sample” decision and no “not sample” decisions, the trace is sampled.
In all other cases, the trace is not sampled.

An “inverted” decision is the one made based on the “invert_match” attribute, such as the one from the string tag policy.

latency block

The latency block configures a policy of type latency. The policy samples based on the duration of the trace. The duration is determined by looking at the earliest start time and latest end time, without taking into consideration what happened in between.

The following arguments are supported:

Name	Type	Description	Default	Required
`threshold_ms`	`number`	Lower latency threshold for sampling, in milliseconds.		yes
`upper_threshold_ms`	`number`	Upper latency threshold for sampling, in milliseconds.	`0`	no

For a trace to be sampled, its latency should be greater than threshold_ms and lower than or equal to upper_threshold_ms.

An upper_threshold_ms of 0 will result in a policy which samples anything greater than threshold_ms.

numeric_attribute block

The numeric_attribute block configures a policy of type numeric_attribute. The policy samples based on number attributes (resource and record).

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	Tag that the filter is matched against.		yes
`min_value`	`number`	The minimum value of the attribute to be considered a match.		yes
`max_value`	`number`	The maximum value of the attribute to be considered a match.		yes
`invert_match`	`bool`	Indicates that values must not match against attribute values.	`false`	no

probabilistic block

The probabilistic block configures a policy of type probabilistic. The policy samples a percentage of traces.

The following arguments are supported:

Name	Type	Description	Default	Required
`sampling_percentage`	`number`	The percentage rate at which traces are sampled.		yes
`hash_salt`	`string`	See below.		no

Use hash_salt to configure the hashing salts. This is important in scenarios where multiple layers of collectors have different sampling rates. If multiple collectors use the same salt with different sampling rates, passing one layer may pass the other even if the collectors have different sampling rates. Configuring different salts avoids that.

status_code block

The status_code block configures a policy of type status_code. The policy samples based upon the status code.

The following arguments are supported:

Name	Type	Description	Default	Required
`status_codes`	`list(string)`	Holds the configurable settings to create a status code filter sampling policy evaluator.		yes

status_codes values must be “OK”, “ERROR” or “UNSET”.

string_attribute block

The string_attribute block configures a policy of type string_attribute. The policy samples based on string attributes (resource and record) value matches. Both exact and regex value matches are supported.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	Tag that the filter is matched against.		yes
`values`	`list(string)`	Set of values or regular expressions to use when matching against attribute values.		yes
`enabled_regex_matching`	`bool`	Determines whether to match attribute values by regexp string.	false	no
`cache_max_size`	`string`	The maximum number of attribute entries of Least Recently Used (LRU) Cache that stores the matched result from the regular expressions defined in `values.`		no
`invert_match`	`bool`	Indicates that values or regular expressions must not match against attribute values.	false	no

rate_limiting block

The rate_limiting block configures a policy of type rate_limiting. The policy samples based on rate.

The following arguments are supported:

Name	Type	Description	Default	Required
`spans_per_second`	`number`	Sets the maximum number of spans that can be processed each second.		yes

span_count block

The span_count block configures a policy of type span_count. The policy samples based on the minimum number of spans within a batch. If all traces within the batch have fewer spans than the threshold, the batch is not sampled.

The following arguments are supported:

Name	Type	Description	Default	Required
`min_spans`	`number`	Minimum number of spans in a trace.		yes
`max_spans`	`number`	Maximum number of spans in a trace.	`0`	no

Set max_spans to 0, if you do not want to limit the policy samples based on the maximum number of spans in a trace.

boolean_attribute block

The boolean_attribute block configures a policy of type boolean_attribute. The policy samples based on a boolean attribute (resource and record).

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	Attribute key to match against.		yes
`value`	`bool`	The bool value (`true` or `false`) to use when matching against attribute values.		yes

ottl_condition block

The ottl_condition block configures a policy of type ottl_condition. The policy samples based on a given boolean OTTL condition (span and span event).

The following arguments are supported:

Name	Type	Description	Default	Required
`error_mode`	`string`	Error handling if OTTL conditions fail to evaluate.		yes
`span`	`list(string)`	OTTL conditions for spans.	`[]`	no
`spanevent`	`list(string)`	OTTL conditions for span events.	`[]`	no

The supported values for error_mode are:

ignore: Ignore errors returned by conditions, log them, and continue on to the next condition. This is the recommended mode.
silent: Ignore errors returned by conditions, do not log them, and continue on to the next condition.
propagate: Return the error up the pipeline. This will result in the payload being dropped from Grafana Agent.

At least one of span or spanevent should be specified. Both span and spanevent can also be specified.

trace_state block

The trace_state block configures a policy of type trace_state. The policy samples based on TraceState value matches.

The following arguments are supported:

Name	Type	Description	Default	Required
`key`	`string`	Tag that the filter is matched against.		yes
`values`	`list(string)`	Set of values to use when matching against trace_state values.		yes

and block

The and block configures a policy of type and. The policy samples based on multiple policies by creating an and policy.

and_sub_policy block

The and_sub_policy block configures a sampling policy used by the and block. At least one and_sub_policy block is required inside an and block.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The custom name given to the policy.		yes
`type`	`string`	The valid policy type for this policy.		yes

composite block

The composite block configures a policy of type composite. This policy samples based on a combination of the above samplers, with ordering and rate allocation per sampler. Rate allocation allocates certain percentages of spans per policy order. For example, if max_total_spans_per_second is set to 100, then rate_allocation is set as follows:

test-composite-policy-1 = 50% of max_total_spans_per_second = 50 spans_per_second
test-composite-policy-2 = 25% of max_total_spans_per_second = 25 spans_per_second
To ensure remaining capacity is filled, use always_sample as one of the policies.

composite_sub_policy block

The composite_sub_policy block configures a sampling policy used by the composite block. At least onecomposite_sub_policy block is required inside a composite block.

The following arguments are supported:

Name	Type	Description	Default	Required
`name`	`string`	The custom name given to the policy.		yes
`type`	`string`	The valid policy type for this policy.		yes

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.tail_sampling is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.tail_sampling does not expose any component-specific debug information.

Example

This example batches trace data from Grafana Agent Flow before sending it to otelcol.exporter.otlp for further processing. This example shows an impractical number of policies for the purpose of demonstrating how to set up each type.

tracing {
  sampling_fraction = 1
  write_to          = [otelcol.processor.tail_sampling.default.input]
}

otelcol.processor.tail_sampling "default" {
  decision_wait               = "10s"
  num_traces                  = 100
  expected_new_traces_per_sec = 10

  policy {
    name = "test-policy-1"
    type = "always_sample"
  }

  policy {
    name = "test-policy-2"
    type = "latency"

    latency {
      threshold_ms = 5000
    }
  }

  policy {
    name = "test-policy-3"
    type = "numeric_attribute"

    numeric_attribute {
      key       = "key1"
      min_value = 50
      max_value = 100
    }
  }

  policy {
    name = "test-policy-4"
    type = "probabilistic"

    probabilistic {
      sampling_percentage = 10
    }
  }

  policy {
    name = "test-policy-5"
    type = "status_code"

    status_code {
      status_codes = ["ERROR", "UNSET"]
    }
  }

  policy {
    name = "test-policy-6"
    type = "string_attribute"

    string_attribute {
      key    = "key2"
      values = ["value1", "value2"]
    }
  }

  policy {
    name = "test-policy-7"
    type = "string_attribute"

    string_attribute {
      key                    = "key2"
      values                 = ["value1", "val*"]
      enabled_regex_matching = true
      cache_max_size         = 10
    }
  }

  policy {
    name = "test-policy-8"
    type = "rate_limiting"

    rate_limiting {
      spans_per_second = 35
    }
  }

  policy {
    name = "test-policy-9"
    type = "string_attribute"

    string_attribute {
      key                    = "http.url"
      values                 = ["/health", "/metrics"]
      enabled_regex_matching = true
      invert_match           = true
    }
  }

  policy {
    name = "test-policy-10"
    type = "span_count"

    span_count {
      min_spans = 2
    }
  }

  policy {
    name = "test-policy-11"
    type = "trace_state"

    trace_state {
      key    = "key3"
      values = ["value1", "value2"]
    }
  }

  policy {
    name = "test-policy-12"
    type = "ottl_condition"
    ottl_condition {
      error_mode = "ignore"
      span = [
        "attributes[\"test_attr_key_1\"] == \"test_attr_val_1\"",
        "attributes[\"test_attr_key_2\"] != \"test_attr_val_1\"",
      ]
      spanevent = [
        "name != \"test_span_event_name\"",
        "attributes[\"test_event_attr_key_2\"] != \"test_event_attr_val_1\"",
      ]
    }
  }

  policy {
    name = "and-policy-1"
    type = "and"

    and {
      and_sub_policy {
        name = "test-and-policy-1"
        type = "numeric_attribute"

        numeric_attribute {
          key       = "key1"
          min_value = 50
          max_value = 100
        }
      }

      and_sub_policy {
        name = "test-and-policy-2"
        type = "string_attribute"

        string_attribute {
          key    = "key1"
          values = ["value1", "value2"]
        }
      }
    }
  }

  policy {
    name = "composite-policy-1"
    type = "composite"

    composite {
      max_total_spans_per_second = 1000
      policy_order               = ["test-composite-policy-1", "test-composite-policy-2", "test-composite-policy-3"]

      composite_sub_policy {
        name = "test-composite-policy-1"
        type = "numeric_attribute"

        numeric_attribute {
          key       = "key1"
          min_value = 50
          max_value = 100
        }
      }

      composite_sub_policy {
        name = "test-composite-policy-2"
        type = "string_attribute"

        string_attribute {
          key    = "key1"
          values = ["value1", "value2"]
        }
      }

      composite_sub_policy {
        name = "test-composite-policy-3"
        type = "always_sample"
      }

      rate_allocation {
        policy  = "test-composite-policy-1"
        percent = 50
      }

      rate_allocation {
        policy  = "test-composite-policy-2"
        percent = 50
      }
    }
  }

  output {
    traces = [otelcol.exporter.otlp.production.input]
  }
}

otelcol.exporter.otlp "production" {
  client {
    endpoint = env("OTLP_SERVER_ENDPOINT")
  }
}

Compatible components

otelcol.processor.tail_sampling can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.tail_sampling has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.processor.transform

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.processor.transform

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.processor.transform accepts telemetry data from other otelcol components and modifies it using the OpenTelemetry Transformation Language (OTTL). OTTL statements consist of OTTL functions, which act on paths. A path is a reference to a telemetry data such as:

Resource attributes.
Instrumentation scope name.
Span attributes.

In addition to the standard OTTL functions, there is also a set of metrics-only functions:

OTTL statements can also contain constructs such as:

Booleans:
- not true
- not IsMatch(name, "http_.*")
Boolean Expressions consisting of a where followed by one or more booleans:
- set(attributes["whose_fault"], "ours") where attributes["http.status"] == 500
- set(attributes["whose_fault"], "theirs") where attributes["http.status"] == 400 or attributes["http.status"] == 404
Math expressions:
- 1 + 1
- end_time_unix_nano - start_time_unix_nano
- sum([1, 2, 3, 4]) + (10 / 1) - 1

Note
There are two ways of inputting strings in River configuration files:

Using quotation marks ( normal River strings). Characters such as \ and " must be escaped by preceding them with a \ character.

Using backticks ( raw River strings). No characters must be escaped. However, it’s not possible to have backticks inside the string.

For example, the OTTL statement set(description, "Sum") where type == "Sum" can be written as:

A normal River string: "set(description, \"Sum\") where type == \"Sum\"".

A raw River string: `set(description, "Sum") where type == "Sum"`.

Raw strings are generally more convenient for writing OTTL statements.

Note
otelcol.processor.transform is a wrapper over the upstream OpenTelemetry Collector transform processor. If necessary, bug reports or feature requests will be redirected to the upstream repository.

You can specify multiple otelcol.processor.transform components by giving them different labels.

Warning
otelcol.processor.transform allows you to modify all aspects of your telemetry. Some specific risks are given below, but this is not an exhaustive list. It is important to understand your data before using this processor.

Unsound Transformations: Transformations between metric data types are not defined in the metrics data model. To use these functions, you must understand the incoming data and know that it can be meaningfully converted to a new metric data type or can be used to create new metrics.

Although OTTL allows you to use the set function with metric.data_type, its implementation in the transform processor is a no-op. To modify a data type, you must use a specific function such as convert_gauge_to_sum.

Identity Conflict: Transformation of metrics can potentially affect a metric’s identity, leading to an Identity Crisis. Be especially cautious when transforming a metric name and when reducing or changing existing attributes. Adding new attributes is safe.

Orphaned Telemetry: The processor allows you to modify span_id, trace_id, and parent_span_id for traces and span_id, and trace_id logs. Modifying these fields could lead to orphaned spans or logs.

Usage

otelcol.processor.transform "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.transform supports the following arguments:

Name	Type	Description	Default	Required
`error_mode`	`string`	How to react to errors if they occur while processing a statement.	`"propagate"`	no

The supported values for error_mode are:

ignore: Ignore errors returned by conditions, log them, and continue on to the next condition. This is the recommended mode.
silent: Ignore errors returned by conditions, do not log them, and continue on to the next condition.
propagate: Return the error up the pipeline. This will result in the payload being dropped from Grafana Agent.

Blocks

The following blocks are supported inside the definition of otelcol.processor.transform:

Hierarchy	Block	Description	Required
trace_statements	trace_statements	Statements which transform traces.	no
metric_statements	metric_statements	Statements which transform metrics.	no
log_statements	log_statements	Statements which transform logs.	no
output	output	Configures where to send received telemetry data.	yes

trace_statements block

The trace_statements block specifies statements which transform trace telemetry signals. Multiple trace_statements blocks can be specified.

Name	Type	Description	Default	Required
`context`	`string`	OTTL Context to use when interpreting the associated statements.		yes
`statements`	`list(string)`	A list of OTTL statements.		yes

The supported values for context are:

resource: Use when interacting only with OTLP resources (for example, resource attributes).
scope: Use when interacting only with OTLP instrumentation scope (for example, the name of the instrumentation scope).
span: Use when interacting only with OTLP spans.
spanevent: Use when interacting only with OTLP span events.

See OTTL Context for more information about how ot use contexts.

metric_statements block

The metric_statements block specifies statements which transform metric telemetry signals. Multiple metric_statements blocks can be specified.

Name	Type	Description	Default	Required
`context`	`string`	OTTL Context to use when interpreting the associated statements.		yes
`statements`	`list(string)`	A list of OTTL statements.		yes

The supported values for context are:

resource: Use when interacting only with OTLP resources (for example, resource attributes).
scope: Use when interacting only with OTLP instrumentation scope (for example, the name of the instrumentation scope).
metric: Use when interacting only with individual OTLP metrics.
datapoint: Use when interacting only with individual OTLP metric data points.

Refer to OTTL Context for more information about how to use contexts.

log_statements block

The log_statements block specifies statements which transform log telemetry signals. Multiple log_statements blocks can be specified.

Name	Type	Description	Default	Required
`context`	`string`	OTTL Context to use when interpreting the associated statements.		yes
`statements`	`list(string)`	A list of OTTL statements.		yes

The supported values for context are:

resource: Use when interacting only with OTLP resources (for example, resource attributes).
scope: Use when interacting only with OTLP instrumentation scope (for example, the name of the instrumentation scope).
log: Use when interacting only with OTLP logs.

See OTTL Context for more information about how ot use contexts.

OTTL Context

Each context allows the transformation of its type of telemetry. For example, statements associated with a resource context will be able to transform the resource’s attributes and dropped_attributes_count.

Each type of context defines its own paths and enums specific to that context. Refer to the OpenTelemetry documentation for a list of paths and enums for each context:

Contexts NEVER supply access to individual items “lower” in the protobuf definition.

This means statements associated to a resource WILL NOT be able to access the underlying instrumentation scopes.
This means statements associated to a scope WILL NOT be able to access the underlying telemetry slices (spans, metrics, or logs).
Similarly, statements associated to a metric WILL NOT be able to access individual datapoints, but can access the entire datapoints slice.
Similarly, statements associated to a span WILL NOT be able to access individual SpanEvents, but can access the entire SpanEvents slice.

For practical purposes, this means that a context cannot make decisions on its telemetry based on telemetry “lower” in the structure. For example, the following context statement is not possible because it attempts to use individual datapoint attributes in the condition of a statement associated to a metric:

metric_statements {
  context = "metric"
  statements = [
    "set(description, \"test passed\") where datapoints.attributes[\"test\"] == \"pass\"",
  ]
}

Context ALWAYS supply access to the items “higher” in the protobuf definition that are associated to the telemetry being transformed.

This means that statements associated to a datapoint have access to a datapoint’s metric, instrumentation scope, and resource.
This means that statements associated to a spanevent have access to a spanevent’s span, instrumentation scope, and resource.
This means that statements associated to a span/metric/log have access to the telemetry’s instrumentation scope, and resource.
This means that statements associated to a scope have access to the scope’s resource.

For example, the following context statement is possible because datapoint statements can access the datapoint’s metric.

metric_statements {
  context = "datapoint"
  statements = [
    "set(metric.description, \"test passed\") where attributes[\"test\"] == \"pass\"",
  ]
}

The protobuf definitions for OTLP signals are maintained on GitHub:

Whenever possible, associate your statements to the context which the statement intens to transform. The contexts are nested, and the higher-level contexts don’t have to iterate through any of the contexts at a lower level. For example, although you can modify resource attributes associated to a span using the span context, it is more efficient to use the resource context.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`input`	`otelcol.Consumer`	A value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.transform is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.transform does not expose any component-specific debug information.

Debug metrics

otelcol.processor.transform does not expose any component-specific debug metrics.

Examples

Perform a transformation if an attribute does not exist

This example sets the attribute test to pass if the attribute test does not exist.

otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "span"
    statements = [
      // Accessing a map with a key that does not exist will return nil.
      `set(attributes["test"], "pass") where attributes["test"] == nil`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each statement is enclosed in backticks instead of quotation marks. This constitutes a raw string, and lets us avoid the need to escape each " with a \" inside a normal River string.

Rename a resource attribute

The are two ways to rename an attribute key. One way is to set a new attribute and delete the old one:

otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "resource"
    statements = [
      `set(attributes["namespace"], attributes["k8s.namespace.name"])`,
      `delete_key(attributes, "k8s.namespace.name")`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Another way is to update the key using regular expressions:

otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "resource"
    statements = [
     `replace_all_patterns(attributes, "key", "k8s\\.namespace\\.name", "namespace")`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each statement is enclosed in backticks instead of quotation marks. This constitutes a raw string, and lets us avoid the need to escape each " with a \", and each \ with a \\ inside a normal River string.

Create an attribute from the contents of a log body

This example sets the attribute body to the value of the log body:

otelcol.processor.transform "default" {
  error_mode = "ignore"

  log_statements {
    context = "log"
    statements = [
      `set(attributes["body"], body)`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each statement is enclosed in backticks instead of quotation marks. This constitutes a raw string, and lets us avoid the need to escape each " with a \" inside a normal River string.

Combine two attributes

This example sets the attribute test to the value of attributes service.name and service.version combined.

otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "resource"
    statements = [
      // The Concat function combines any number of strings, separated by a delimiter.
      `set(attributes["test"], Concat([attributes["foo"], attributes["bar"]], " "))`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each statement is enclosed in backticks instead of quotation marks. This constitutes a raw string, and lets us avoid the need to escape each " with a \" inside a normal River string.

Parsing JSON logs

Given the following JSON body:

{
  "name": "log",
  "attr1": "example value 1",
  "attr2": "example value 2",
  "nested": {
    "attr3": "example value 3"
  }
}

You can add specific fields as attributes on the log:

otelcol.processor.transform "default" {
  error_mode = "ignore"

  log_statements {
    context = "log"

    statements = [
      // Parse body as JSON and merge the resulting map with the cache map, ignoring non-json bodies.
      // cache is a field exposed by OTTL that is a temporary storage place for complex operations.
      `merge_maps(cache, ParseJSON(body), "upsert") where IsMatch(body, "^\\{")`,

      // Set attributes using the values merged into cache.
      // If the attribute doesn't exist in cache then nothing happens.
      `set(attributes["attr1"], cache["attr1"])`,
      `set(attributes["attr2"], cache["attr2"])`,

      // To access nested maps you can chain index ([]) operations.
      // If nested or attr3 do no exist in cache then nothing happens.
      `set(attributes["nested.attr3"], cache["nested"]["attr3"])`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Various transformations of attributes and status codes

The example takes advantage of context efficiency by grouping transformations with the context which it intends to transform.

otelcol.receiver.otlp "default" {
  http {}
  grpc {}

  output {
    metrics = [otelcol.processor.transform.default.input]
    logs    = [otelcol.processor.transform.default.input]
    traces  = [otelcol.processor.transform.default.input]
  }
}

otelcol.processor.transform "default" {
  error_mode = "ignore"

  trace_statements {
    context = "resource"
    statements = [
      `keep_keys(attributes, ["service.name", "service.namespace", "cloud.region", "process.command_line"])`,
      `replace_pattern(attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")`,
      `limit(attributes, 100, [])`,
      `truncate_all(attributes, 4096)`,
    ]
  }

  trace_statements {
    context = "span"
    statements = [
      `set(status.code, 1) where attributes["http.path"] == "/health"`,
      `set(name, attributes["http.route"])`,
      `replace_match(attributes["http.target"], "/user/*/list/*", "/user/{userId}/list/{listId}")`,
      `limit(attributes, 100, [])`,
      `truncate_all(attributes, 4096)`,
    ]
  }

  metric_statements {
    context = "resource"
    statements = [
      `keep_keys(attributes, ["host.name"])`,
      `truncate_all(attributes, 4096)`,
    ]
  }

  metric_statements {
    context = "metric"
    statements = [
      `set(description, "Sum") where type == "Sum"`,
    ]
  }

  metric_statements {
    context = "datapoint"
    statements = [
      `limit(attributes, 100, ["host.name"])`,
      `truncate_all(attributes, 4096)`,
      `convert_sum_to_gauge() where metric.name == "system.processes.count"`,
      `convert_gauge_to_sum("cumulative", false) where metric.name == "prometheus_metric"`,
    ]
  }

  log_statements {
    context = "resource"
    statements = [
      `keep_keys(attributes, ["service.name", "service.namespace", "cloud.region"])`,
    ]
  }

  log_statements {
    context = "log"
    statements = [
      `set(severity_text, "FAIL") where body == "request failed"`,
      `replace_all_matches(attributes, "/user/*/list/*", "/user/{userId}/list/{listId}")`,
      `replace_all_patterns(attributes, "value", "/account/\\d{4}", "/account/{accountId}")`,
      `set(body, attributes["http.route"])`,
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = env("OTLP_ENDPOINT")
  }
}

Compatible components

otelcol.processor.transform can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

otelcol.processor.transform has exports that can be consumed by the following components:

Components that consume OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

otelcol.receiver.jaeger

Fri, 10 Apr 2026 07:58:08 +0000

otelcol.receiver.jaeger

otelcol.receiver.jaeger accepts Jaeger-formatted data over the network and forwards it to other otelcol.* components.

NOTE: otelcol.receiver.jaeger is a wrapper over the upstream OpenTelemetry Collector jaeger receiver. Bug reports or feature requests will be redirected to the upstream repository, if necessary.

Multiple otelcol.receiver.jaeger components can be specified by giving them different labels.

Usage

otelcol.receiver.jaeger "LABEL" {
  protocols {
    grpc {}
    thrift_http {}
    thrift_binary {}
    thrift_compact {}
  }

  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.receiver.jaeger doesn’t support any arguments and is configured fully through inner blocks.

Blocks

The following blocks are supported inside the definition of otelcol.receiver.jaeger:

Hierarchy	Block	Description	Required
protocols	protocols	Configures the protocols the component can accept traffic over.	yes
protocols > grpc	grpc	Configures a Jaeger gRPC server to receive traces.	no
protocols > grpc > tls	tls	Configures TLS for the gRPC server.	no
protocols > grpc > keepalive	keepalive	Configures keepalive settings for the configured server.	no
protocols > grpc > keepalive > server_parameters	server_parameters	Server parameters used to configure keepalive settings.	no
protocols > grpc > keepalive > enforcement_policy	enforcement_policy	Enforcement policy for keepalive settings.	no
protocols > thrift_http	thrift_http	Configures a Thrift HTTP server to receive traces.	no
protocols > thrift_http > tls	tls	Configures TLS for the Thrift HTTP server.	no
protocols > thrift_http > cors	cors	Configures CORS for the Thrift HTTP server.	no
protocols > thrift_binary	thrift_binary	Configures a Thrift binary UDP server to receive traces.	no
protocols > thrift_compact	thrift_compact	Configures a Thrift compact UDP server to receive traces.	no
debug_metrics	debug_metrics	Configures the metrics that this component generates to monitor its state.	no
output	output	Configures where to send received telemetry data.	yes

The > symbol indicates deeper levels of nesting. For example, protocols > grpc refers to a grpc block defined inside a protocols block.

protocols block

The protocols block defines a set of protocols that will be used to accept traces over the network.

protocols doesn’t support any arguments and is configured fully through inner blocks.

otelcol.receiver.jeager requires at least one protocol block (grpc, thrift_http, thrift_binary, or thrift_compact) to be provided.

grpc block

The grpc block configures a gRPC server which can accept Jaeger traces. If the grpc block isn’t provided, a gRPC server isn’t started.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:14250"`	no
`transport`	`string`	Transport to use for the gRPC server.	`"tcp"`	no
`max_recv_msg_size`	`string`	Maximum size of messages the server will accept. 0 disables a limit.		no
`max_concurrent_streams`	`number`	Limit the number of concurrent streaming RPC calls.		no
`read_buffer_size`	`string`	Size of the read buffer the gRPC server will use for reading from clients.	`"512KiB"`	no
`write_buffer_size`	`string`	Size of the write buffer the gRPC server will use for writing to clients.		no
`include_metadata`	`boolean`	Propagate incoming connection metadata to downstream consumers.		no

tls block

The tls block configures TLS settings used for a server. If the tls block isn’t provided, TLS won’t be used for connections to the server.

The following arguments are supported:

Name	Type	Description	Default	Required
`ca_file`	`string`	Path to the CA file.		no
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.		no
`cert_file`	`string`	Path to the TLS certificate.		no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.		no
`insecure_skip_verify`	`boolean`	Ignores insecure server TLS certificates.		no
`include_system_ca_certs_pool`	`boolean`	Whether to load the system certificate authorities pool alongside the certificate authority.	`false`	no
`insecure`	`boolean`	Disables TLS when connecting to the configured server.		no
`key_file`	`string`	Path to the TLS certificate key.		no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.		no
`max_version`	`string`	Maximum acceptable TLS version for connections.	`"TLS 1.3"`	no
`min_version`	`string`	Minimum acceptable TLS version for connections.	`"TLS 1.2"`	no
`cipher_suites`	`list(string)`	A list of TLS cipher suites that the TLS transport can use.	`[]`	no
`reload_interval`	`duration`	The duration after which the certificate is reloaded.	`"0s"`	no
`server_name`	`string`	Verifies the hostname of server certificates when set.		no

If the server doesn’t support TLS, you must set the insecure argument to true.

To disable tls for connections to the server, set the insecure argument to true.

If reload_interval is set to "0s", the certificate never reloaded.

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

If cipher_suites is left blank, a safe default list is used. See the Go TLS documentation for a list of supported cipher suites.

keepalive block

The keepalive block configures keepalive settings for connections to a gRPC server.

keepalive doesn’t support any arguments and is configured fully through inner blocks.

server_parameters block

The server_parameters block controls keepalive and maximum age settings for gRPC servers.

The following arguments are supported:

Name	Type	Description	Default	Required
`max_connection_idle`	`duration`	Maximum age for idle connections.	`"infinity"`	no
`max_connection_age`	`duration`	Maximum age for non-idle connections.	`"infinity"`	no
`max_connection_age_grace`	`duration`	Time to wait before forcibly closing connections.	`"infinity"`	no
`time`	`duration`	How often to ping inactive clients to check for liveness.	`"2h"`	no
`timeout`	`duration`	Time to wait before closing inactive clients that do not respond to liveness checks.	`"20s"`	no

enforcement_policy block

The enforcement_policy block configures the keepalive enforcement policy for gRPC servers. The server will close connections from clients that violate the configured policy.

The following arguments are supported:

Name	Type	Description	Default	Required
`min_time`	`duration`	Minimum time clients should wait before sending a keepalive ping.	`"5m"`	no
`permit_without_stream`	`boolean`	Allow clients to send keepalive pings when there are no active streams.	`false`	no

thrift_http block

The thrift_http block configures an HTTP server which can accept Thrift-formatted traces. If the thrift_http block isn’t specified, an HTTP server isn’t started.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:14268"`	no
`max_request_body_size`	`string`	Maximum request body size the server will allow.	`20MiB`	no
`include_metadata`	`boolean`	Propagate incoming connection metadata to downstream consumers.		no

cors block

The cors block configures CORS settings for an HTTP server.

The following arguments are supported:

Name	Type	Description	Default	Required
`allowed_origins`	`list(string)`	Allowed values for the `Origin` header.		no
`allowed_headers`	`list(string)`	Accepted headers from CORS requests.	`["X-Requested-With"]`	no
`max_age`	`number`	Configures the `Access-Control-Max-Age` response header.		no

The allowed_headers specifies which headers are acceptable from a CORS request. The following headers are always implicitly allowed:

Accept
Accept-Language
Content-Type
Content-Language

If allowed_headers includes "*", all headers will be permitted.

thrift_binary block

The thrift_binary block configures a UDP server which can accept traces formatted to the Thrift binary protocol. If the thrift_binary block isn’t provided, a UDP server isn’t started.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:6832"`	no
`queue_size`	`number`	Maximum number of UDP messages that can be queued at once.	`1000`	no
`max_packet_size`	`string`	Maximum UDP message size.	`"65KiB"`	no
`workers`	`number`	Number of workers to concurrently read from the message queue.	`10`	no
`socket_buffer_size`	`string`	Buffer to allocate for the UDP socket.		no

thrift_compact block

The thrift_compact block configures a UDP server which can accept traces formatted to the Thrift compact protocol. If the thrift_compact block isn’t provided, a UDP server isn’t started.

The following arguments are supported:

Name	Type	Description	Default	Required
`endpoint`	`string`	`host:port` to listen for traffic on.	`"0.0.0.0:6831"`	no
`queue_size`	`number`	Maximum number of UDP messages that can be queued at once.	`1000`	no
`max_packet_size`	`string`	Maximum UDP message size.	`"65KiB"`	no
`workers`	`number`	Number of workers to concurrently read from the message queue.	`10`	no
`socket_buffer_size`	`string`	Buffer to allocate for the UDP socket.		no

debug_metrics block

The debug_metrics block configures the metrics that this component generates to monitor its state.

The following arguments are supported:

Name	Type	Description	Default	Required
`disable_high_cardinality_metrics`	`boolean`	Whether to disable certain high cardinality metrics.	`true`	no

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

Name	Type	Description	Default	Required
`logs`	`list(otelcol.Consumer)`	List of consumers to send logs to.	`[]`	no
`metrics`	`list(otelcol.Consumer)`	List of consumers to send metrics to.	`[]`	no
`traces`	`list(otelcol.Consumer)`	List of consumers to send traces to.	`[]`	no

Exported fields

otelcol.receiver.jaeger does not export any fields.

Component health

otelcol.receiver.jaeger is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.receiver.jaeger does not expose any component-specific debug information.

Example

This example creates a pipeline which accepts Jaeger-formatted traces and writes them to an OTLP server:

otelcol.receiver.jaeger "default" {
  protocols {
    grpc {}
    thrift_http {}
    thrift_binary {}
    thrift_compact {}
  }

  output {
    traces = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    traces = [otelcol.exporter.otlp.default.input]
  }
}

otelcol.exporter.otlp "default" {
  client {
    endpoint = "my-otlp-server:4317"
  }
}

Technical details

otelcol.receiver.jaeger supports gzip for compression.

Compatible components

otelcol.receiver.jaeger can accept arguments from the following components:

Components that export OpenTelemetry otelcol.Consumer

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

Components reference on Grafana Labs

discovery.azure

discovery.azure

Usage

Arguments

Blocks

oauth block

managed_identity block

tls_config block

Exported fields

Component health

Debug information

Debug metrics

Example

Compatible components

discovery.consul

discovery.consul

Usage

Arguments

Blocks

basic_auth block

authorization block

oauth2 block

tls_config block

Exported fields

Component health

Debug information

Debug metrics

Example

Compatible components

discovery.consulagent

discovery.consulagent

Usage

Arguments

Blocks

tls_config block

Exported fields

Component health

Debug information

Debug metrics

Example

Compatible components

discovery.digitalocean

discovery.digitalocean

Usage

Arguments

Blocks

Exported fields

Component health

Debug information

Debug metrics

Example

Compatible components

discovery.dns

discovery.dns

Usage

Arguments

Exported fields

Component health

Debug information

Debug metrics

Example

Compatible components

discovery.docker

discovery.docker

Usage

Arguments

Blocks

filter block

basic_auth block

authorization block

oauth2 block

tls_config block

Exported fields

Component health

Debug information

Debug metrics

Examples

Linux or macOS hosts

Windows hosts