Grafana Cloud Instrument and send data OpenTelemetry Protocol OTLP format considerations

OpenTelemetry Protocol (OTLP) format considerations

This section describes metrics and logs conversion considerations for data sent using OpenTelemetry Protocol (OTLP). For a guide to get started, see Send data using OpenTelemetry Protocol.

Metrics

We store the metrics in a Prometheus-compatible store. We currently follow the official specification described here: OTLP Metric points to Prometheus.

By default, Grafana Cloud does not accept OpenTelemetry Exponential Histograms. For Grafana Cloud to accept them, ingestion of Prometheus Native Histogram metrics must first be enabled in your environment. To enable such metrics, see Send Prometheus Native Histograms. After this is done, Grafana Cloud will accept OpenTelemetry Exponential Histograms, and convert them into Prometheus Native Histograms following the conventions described in the Exponential Histograms specification.

The following sections describe other common issues you might experience.

Dots (.) are converted to _

Because Prometheus metrics don’t support . and - in metric or label names, they are converted to _ in Prometheus.

For example:

requests.duration{http.status_code=500, cloud.region=us-central1} in OTLP becomes requests_duration{http_status_code=”500”, cloud_region=”us-central1”} in Prometheus.

Resource attributes are added to target_info metric

As described in the Resource Attributes specification, we add resource attributes to a special target_info metric.

However, <service.namespace>/<service.name> or <service.name> (if namespace is empty), is added as job label and service.instance.id is added as instance label to every metric.

Logs

Because it is too costly from a cardinality perspective, Grafana Loki indexes a few attributes from log entries instead of indexing all available attributes or the entire log message. As such, you must provide hints to the Loki translator, stating which attributes to promote to Loki labels. You can do this by adding new synthetic attributes, which are read by the Loki translator and removed before the data is sent over the network.

The following snippet shows how the processors section looks when you add a resource processor that adds the loki.resource.labels hint. This example tells the Loki translator that the host_name resource attribute should be promoted to a label. You are not required to add labels, and every entry that passes through the Loki exporter will have a static label exporter with the value OTLP by default.

For more information about labels and how to chose the right ones for your use case, refer to the Loki documentation.

yaml Copy

processors:
  resource:
    attributes:
      # this is how we tell Loki to add `host_name` as a label
      - action: insert
        key: loki.resource.labels
        value: host_name

      # this would ideally come from the data source itself, but we set it here
      # for simplicity
      - action: insert
        key: host_name
        value: guarana