Migration of logs from Loki Exporter to Grafana Loki native OTLP format

Note
If you need help switching to the native OTLP format or assess which format you are using, contact support.

Grafana Cloud introduced native OTLP ingestion support in Loki 3.x on March 25, 2024. The new format aligns with the OTLP specification and offers an improved formatting and querying experience.

Loki Exporter has been deprecated as of July 2024. It is highly advised to migrate your logs to the native format for all the benefits mentioned in the next section. This guide is for Grafana Cloud users who are using the Loki Exporter format, and covers the benefits of the native OTLP format and how to migrate.

Benefits of native OTLP support

Future improvements: The native OTLP ingestion support for Loki 3 represents the future of log ingestion. Upgrading to the native OTLP format ensures that you benefit from the latest enhancements and have the best possible user experience.
Simplified client config: LokiExporter requires setting hints for managing stream labels. With the native OTLP format there is no added complexity in setting hints for your client to manage labels.
Simplified querying: Loki native OTLP ingestion leverages the use of structured metadata to store resource and log attributes. This allows you to interact with attributes without having to use parsers in queries which can leverage query acceleration to increase query performance.
Better data model: Better aligned with modern observability practices than the older JSON based model.

Format differences

The following summary describes the main differences between LokiExporter and native OTLP ingestion support for Loki 3:

LokiExporter:

Index labels:
- Supports label control via hints set by the OpenTelemetry client/collector.
- Default labels:
  - job=LogRecord.Resource.Attributes["service.namespace"]/LogRecord.Resource.Attributes["service.name"]
  - instance=LogRecord.Resource.Attributes["service.instance.id"]
  - exporter=“OTLP”
  - level=LogRecord.SeverityText
Log Body: encodes log records and attributes in json(default) or logfmt format.

Native OTLP support:

Index Labels:
- Pre-configured resource attributes are picked as index labels, to learn more refer to the OTLP format considerations documentation.
- Custom attributes can be promoted to index labels via per-tenant OTLP configuration in Loki.
Log Line: LogRecord.Body as a string
Structured Metadata: Anything not stored in index labels and LogLine gets stored as structured metadata.

Sample log

The following example demonstrates how the sample log record would appear in both formats after ingestion with the default configuration:

yaml
- Resource Attributes:
    - service.name: 'shoppingcart'
    - service.namespace: 'shop'
    - deployment.environment: 'staging'
- InstrumentationScope:
    - Name: 'login'
- LogRecord:
    - Timestamp: 1715247552000000000
    - Body: 'user logged in'
    - SeverityText: 'INFO'
    - SeverityNumber: 9
    - TraceId: '08040201000000000000000000000000'
    - SpanId: '0102040800000000'
    - Attributes:
        - thread.name: 'main'

LokiExporter:

yaml
- Index Labels:
  - job="shop/shoppingcart"
  - exporter="OTLP"
- Log Body: `{"body":"user logged in","severity_text":"INFO","attributes":{"traceid":"08040201000000000000000000000000", "spanid": "0102040800000000"}, "resources":{"deployment.environment": "staging","service.name":"shoppingcart","service.namespace":"shop"}, "instrumentation_scope": {"name":"login"}}`

Native OTLP support:

yaml
- Index Labels:
    - deployment_environment="staging"
    - service_name="shoppingcart"
    - service_namespace="shop"
- Log Body: 'user logged in'
- Structured Metadata:
    - severity_text: 'INFO'
    - severity_number: '9'
    - trace_id: '08040201000000000000000000000000'
    - span_id: '0102040800000000'
    - scope_name: 'login'

Query experience

The LokiExporter encodes data to json or logfmt blob, that has to be decoded at query time to interact with OpenTelemetry attributes. Loki 3 with native OTLP ingestion doesn’t encode data before storing it. It leverages structured metadata, allowing you to interact with attributes without having to use parsers in queries.

The following examples compare the querying experience of the two formats using the previous log line example:

Query all logs without any filters:

LokiExporter: {job="shop/shoppingcart"}
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"}

Query logs with severity_text=INFO:

LokiExporter: {job="shop/shoppingcart"} | json | severity_text="INFO"
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"} | severity_text="INFO"

Query logs with non-indexed resource attribute and deployment_environment="staging":

LokiExporter: {job="shop/shoppingcart"} | json | resources_deployment_environment="staging"
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"} | deployment_environment="staging"

Query logs with specific TraceId:

LokiExporter: {job="shop/shoppingcart"} | json | attributes_traceid="08040201000000000000000000000000"
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"} | trace_id="08040201000000000000000000000000"

Query logs with InstrumentationScope.Name="login":

LokiExporter: {job="shop/shoppingcart"} | json | instrumentation_scope_name="login"
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"} | scope_name="login"

Query logs with LogRecord.Attributes["thread.name"]="main":

LokiExporter: {job="shop/shoppingcart"} | json | attributes_thread_name="name"
Native OTLP support: {service_name="shoppingcart", service_namespace="shop"} | thread_name="main"

Display log message as log line in logs query results:

LokiExporter: {job="shop/shoppingcart"} | json | line_format "{{.body}}"
native OTLP support: {service_name="shoppingcart", service_namespace="shop"}

Count logs in the last hour by severity:

LokiExporter: sum(count_over_time({job="shop/shoppingcart"} | json[1h])) by (severity_text)
native OTLP support: sum(count_over_time({service_name="shoppingcart", service_namespace="shop"}[1h])) by (severity_text)

Switch to native OTLP ingestion support for Loki

Rewrite all your LogQL queries to query logs according to the new format.
Reach out to support if you need to update the OTLP configuration for your tenant. Refer to the Loki OTLP documentation for information on resource attributes and index labels.
Contact support to unpin your tenant from the LokiExporter format and start using the native OTLP format.
If you use Application Observability, update your logs configuration from Loki exporter query to OTLP gateway / native Loki otlp query.
Perform a visualization migration in Grafana to be able to seamlessly navigate between logs and traces.