OpenTelemetry Collector

Set up OpenTelemetry Collector for Application Observability

The OpenTelemetry project maintainers and the Cloud Native Computing Foundation maintain the upstream OpenTelemetry Collector.

For production observability, Grafana Labs recommends Grafana Alloy, an OpenTelemetry Collector distribution that packages upstream OpenTelemetry Collector components and Prometheus exporters. Alloy provides stability, support, and an integrated experience with Grafana Application Observability and other Grafana products.

Recommended setup

For production, run an OpenTelemetry Collector on every host to seamlessly correlate data between infrastructure and application observability.

Grafana Labs recommends using the OpenTelemetry Collector Grafana Cloud integration tile to configure the OpenTelemetry Collector.

If you already have an OpenTelemetry Collector per host, refer to the advanced manual setup and configure your application sections.

Advanced manual setup

For advanced use cases, manually configure the OpenTelemetry config.yaml configuration file:

# Tested with OpenTelemetry Collector Contrib v0.98.0
receivers:
  otlp:
    # https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver
    protocols:
      grpc:
      http:
  hostmetrics:
    # Optional. Host Metrics Receiver added as an example of Infra Monitoring capabilities of the OpenTelemetry Collector
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver
    scrapers:
      load:
      memory:

processors:
  batch:
    # https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor
  resourcedetection:
    # Enriches telemetry data with resource information from the host
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor
    detectors: ["env", "system"]
    override: false
  transform/drop_unneeded_resource_attributes:
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - delete_key(attributes, "k8s.pod.start_time")
          - delete_key(attributes, "os.description")
          - delete_key(attributes, "os.type")
          - delete_key(attributes, "process.command_args")
          - delete_key(attributes, "process.executable.path")
          - delete_key(attributes, "process.pid")
          - delete_key(attributes, "process.runtime.description")
          - delete_key(attributes, "process.runtime.name")
          - delete_key(attributes, "process.runtime.version")
    metric_statements:
      - context: resource
        statements:
          - delete_key(attributes, "k8s.pod.start_time")
          - delete_key(attributes, "os.description")
          - delete_key(attributes, "os.type")
          - delete_key(attributes, "process.command_args")
          - delete_key(attributes, "process.executable.path")
          - delete_key(attributes, "process.pid")
          - delete_key(attributes, "process.runtime.description")
          - delete_key(attributes, "process.runtime.name")
          - delete_key(attributes, "process.runtime.version")
    log_statements:
      - context: resource
        statements:
          - delete_key(attributes, "k8s.pod.start_time")
          - delete_key(attributes, "os.description")
          - delete_key(attributes, "os.type")
          - delete_key(attributes, "process.command_args")
          - delete_key(attributes, "process.executable.path")
          - delete_key(attributes, "process.pid")
          - delete_key(attributes, "process.runtime.description")
          - delete_key(attributes, "process.runtime.name")
          - delete_key(attributes, "process.runtime.version")
  transform/add_resource_attributes_as_metric_attributes:
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
    error_mode: ignore
    metric_statements:
      - context: datapoint
        statements:
          - set(attributes["deployment.environment"], resource.attributes["deployment.environment"])
          - set(attributes["service.version"], resource.attributes["service.version"])

exporters:
  otlphttp/grafana_cloud:
    # https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter
    endpoint: "${env:GRAFANA_CLOUD_OTLP_ENDPOINT}"
    auth:
      authenticator: basicauth/grafana_cloud

extensions:
  basicauth/grafana_cloud:
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/basicauthextension
    client_auth:
      username: "${env:GRAFANA_CLOUD_INSTANCE_ID}"
      password: "${env:GRAFANA_CLOUD_API_KEY}"

service:
  extensions:
    [
      basicauth/grafana_cloud,
    ]
  telemetry:
    metrics:
      level: detailed
      address: 0.0.0.0:8888
  pipelines:
    traces:
      receivers: [otlp]
      processors:
        [resourcedetection, transform/drop_unneeded_resource_attributes, batch]
      exporters: [otlphttp/grafana_cloud]
    metrics:
      receivers: [otlp, hostmetrics]
      processors:
        [
          resourcedetection,
          transform/drop_unneeded_resource_attributes,
          transform/add_resource_attributes_as_metric_attributes,
          batch,
        ]
      exporters: [otlphttp/grafana_cloud]
    logs:
      receivers: [otlp]
      processors:
        [
          resourcedetection,
          transform/drop_unneeded_resource_attributes,
          batch,
        ]
      exporters: [otlphttp/grafana_cloud]

Set the following environmental variables in the configuration file:

Environment Variable	Description	Example
`GRAFANA_CLOUD_API_KEY`	API key generated above	`eyJvSomeLongStringJ9fQ==`
`GRAFANA_CLOUD_OTLP_ENDPOINT`	OTLP endpoint from Grafana Cloud > OpenTelemetry > Configure	`https://otlp-endpoint-xyz.grafana.net/otlp`
`GRAFANA_CLOUD_INSTANCE_ID`	Instance ID from Grafana Cloud > OpenTelemetry > Configure

Host identification for Application Observability

Application Observability identifies hosts automatically from standard OpenTelemetry resource attributes attached to your telemetry. The minimal configuration above doesn’t include explicit host identifier configuration because proper host identification depends on your deployment environment.

Application Observability evaluates the following resource attributes in priority order and uses the first match as the host’s billing identifier:

Priority	Attributes	Typical environment
1	`k8s.node.name`	Kubernetes
2	`host.name` together with `cloud.provider`	Cloud VMs
3	`grafana.host.id`	Bare metal, Docker

For environment-specific configuration guidance, refer to the Application Observability host-hours pricing documentation, which provides detailed examples for:

Kubernetes deployments using the k8sattributes processor
Cloud VM deployments with cloud provider detection
Bare metal and Docker deployments using grafana.host.id

Note
If host identification isn’t configured correctly for your environment, Application Observability will guide you to the appropriate resolution within the product.

Collector self-monitoring

The collector telemetry configuration enables monitoring of the collector itself. This exposes collector performance metrics at :8888/metrics including pipeline throughput, queue sizes, and resource usage. Use these metrics to troubleshoot collector performance issues and ensure your observability pipeline is healthy.

Data pipelines

OpenTelemetry Collector receives OTLP data and processes it with the following pipelines:

Traces:

The traces pipeline receives traces with the otlp receiver and exports them to the Grafana Cloud Tempo with the otlp exporter.

The traces pipeline uses the resourcedetection processor to enrich telemetry data with resource information from the host.

Consult the resource detection processor README.md for a list of configuration options.

Metrics:

The metrics pipeline receives traces from the otlp receiver and exports metrics to the Grafana Cloud Metrics with the prometheusremotewrite exporter.

The metrics pipeline uses the transform processor to add deployment.environment, and service.version labels to metrics.

Logs:

The logs pipeline receives logs with the otlp receiver and exports them to the Grafana Cloud Loki with the loki exporter.

Run OpenTelemetry Collector

Create the config.yaml file, set the necessary environment variables, and run the OpenTelemetry Collector.

Configure your application

Set the following environment variables to configure your application to use the OpenTelemetry Collector:

Configuration	Options	Result
`export OTEL_EXPORTER_OTLP_ENDPOINT=<host>`	`http://localhost:4318`, remote host address	The default local host address, or a remote host address.
`export OTEL_EXPORTER_OTLP_PROTOCOL=<protocol>`	`grpc`, `http/protobuf`	The default `http/protobuf` protocol or `grpc`

For example, for a local OpenTelemetry Collector set the following environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc

Then restart your application.

Troubleshoot host identification

If Application Observability displays a notification that host information isn’t being received, verify that at least one of the priority tiers is satisfied:

Check resource attributes: Use Explore to query your service’s spans and verify which host attributes are present:
```
{service.name="your-service"} | json
```
Look for k8s.node.name, host.name with cloud.provider, or grafana.host.id in the resource attributes.
Verify resourcedetection processor: Ensure the resourcedetection processor is active in your traces pipeline and configured to detect host.id and host.name.
For Kubernetes: Add the k8sattributes processor to stamp k8s.node.name on your telemetry.
For cloud VMs: Enable cloud-specific resource detection (AWS EC2, GCP, Azure) to set both host.name and cloud.provider.
For bare metal/Docker: Explicitly set grafana.host.id using the transform processor.

For detailed configuration examples for each environment, refer to the Application Observability host-hours pricing documentation.

Next steps

Observe your services in Application Observability

Was this page helpful?

Email docs@grafana.com

Help and support

Community

Set up OpenTelemetry Collector for Application Observability

Recommended setup

Advanced manual setup

Host identification for Application Observability

Collector self-monitoring

Data pipelines

Run OpenTelemetry Collector

Configure your application

Troubleshoot host identification

Next steps

Was this page helpful?

Still have questions?

Get every update