This is documentation for the next version of Mimir. For the latest stable release, go to the latest version.

Documentation Grafana Mimir Configure OpenTelemetry Collector

Configure the OpenTelemetry Collector to write metrics into Mimir

When using the OpenTelemetry Collector, you can use the OpenTelemetry protocol (OTLP) or the Prometheus remote write protocol to write metrics into Mimir. It’s recommended that you use the OpenTelemetry protocol.

Use the OpenTelemetry protocol

Mimir supports native OTLP over HTTP. To configure the collector to use the OTLP interface, use the otlphttp exporter and the native Mimir endpoint. For example:

yaml Copy

exporters:
  otlphttp:
    endpoint: http://<mimir-endpoint>/otlp

Then, enable it in the service.pipelines block:

yaml Copy

service:
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]

If you want to authenticate using basic auth, use the basicauth extension. For example:

yaml Copy

extensions:
  basicauth/otlp:
    client_auth:
      username: username
      password: password

exporters:
  otlphttp:
    auth:
      authenticator: basicauth/otlp
    endpoint: http://<mimir-endpoint>/otlp

service:
  extensions: [basicauth/otlp]
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]

Use the Prometheus remote write protocol

To use the Prometheus remote write protocol to send metrics into Mimir, use the prometheusremotewrite exporter in the Collector and the native Mimir endpoint.

In the exporters section, add:

yaml Copy

exporters:
  prometheusremotewrite:
    endpoint: http://<mimir-endpoint>/api/v1/push

Then, enable it in the service.pipelines block:

yaml Copy

service:
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., prometheusremotewrite]

If you want to authenticate using basic auth, use the basicauth extension. For example:

yaml Copy

extensions:
  basicauth/prw:
    client_auth:
      username: username
      password: password

exporters:
  prometheusremotewrite:
    auth:
      authenticator: basicauth/prw
    endpoint: http://<mimir-endpoint>/api/v1/push

service:
  extensions: [basicauth/prw]
  pipelines:
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., prometheusremotewrite]

Format considerations

We follow the official OTLP Metric points to Prometheus specification.

By default, Grafana Mimir does not accept OpenTelemetry Exponential Histogram metrics. For Grafana Mimir to accept them, ingestion of Prometheus Native Histogram metrics must first be enabled following the instructions in Configure native histogram ingestion. After this is done, Grafana Mimir will accept OpenTelemetry Exponential Histograms, and convert them into Prometheus Native Histograms following the conventions described in the Exponential Histograms specification.

You might experience the following common issues:

• Dots (.) are converted to _

  Prometheus metrics do not support . and - characters in metric or label names. Prometheus converts these characters to _.

  For example:

  requests.duration{http.status_code=500, cloud.region=us-central1} in OTLP

  requests_duration{http_status_code=”500”, cloud_region=”us-central1”} in Prometheus

• Resource attributes are added to the target_info metric.

  However, <service.namespace>/<service.name> or <service.name> (if the namespace is empty), is added as the label job, and service.instance.id is added as the label instance to every metric.

  For details, see the OpenTelemetry Resource Attributes specification.