Explore OpenTelemetry Logs in Loki
Once you have data being sent to your Grafana Cloud or OSS stack, you can view log data via Grafana’s Explore UI. This document focuses on launching Explore from a Grafana Cloud stack, but navigation for an OSS stack should be similar.
To explore OpenTelemetry data, go to your Grafana Cloud console, select your Grafana instance and click Launch.
Then select Explore from the left side menu.
Once Explore opens, select your Loki instance from the dropdown menu.
When using GrafanaCloud, the naming convention is grafanacloud-<STACK_NAME>-log.
To view the OpenTelemetry logs in Loki, you can either use the Builder or Code tool to construct a query.
Every query must start with a stream selector. Select one or more labels to create the stream selector. An example of a simple one
follows.
{exporter="OTLP"}
If you run the above query for an appropriate time window, you will see all logs that were sent via an OTLP exporter.
When viewing the OpenTelemetry log entries in Loki, the Prettify Json option can be toggled on to help make large statements easier to
read; however, you can also write a query so that only fields of interests are included. To do this, add a
json stage to the query to transform the log statement into key:value pairs.
{exporter="OTLP"} | json
Run the above query and then scroll down to the result set. Click on a record to expand and view the key:value pairs.
Next, you can customize the Loki log format with a line_format expression that includes only fields of interest.
{exporter="OTLP"} | json | line_format "[{{ .attributes_thread_name }}] {{ .body}}"
The above query will create a result set that includes only the time, thread name and message body.
At the time of writing, the thread name is still an experimental attribute and has to be enabled explicitly (e.g. for the Java Agent or Spring Starter).
The logger name is also a very common attribute ({{ .instrumentation_scope_name }}) - but cannot be used with
Grafana Agent at the time of writing.
Common Framework Formats
The following table provides line_format expressions that attempt to align with a given framework’s default logging format.
We suggest to replace the process ID (resources_process_pid) with the service.instance.id, which makes it easy to identify a faulty node.
| Framework | Format pattern in Framework | Format pattern in Grafana | LogQL expression |
|---|---|---|---|
| Java | TIMESTAMP SEVERITY PROCESS_PID --- [THREAD] LOGGING CLASS : MESSAGE | TIMESTAMP SEVERITY [SERVICE_INSTANCE_ID/THREAD NAME] - LOGGER NAME: MESSAGE | {exporter="OTLP"} | json | line_format "{{if .severity}}{{alignRight 5 .severity}}{{end}} {{alignRight 10 .resources_service_instance_id}}{{if .attributes_thread_name}}/{{alignRight 20 .attributes_thread_name}}{{end}}{{if .instrumentation_scope_name }} - {{alignRight 40 .instrumentation_scope_name}}{{end}}: {{.body}}" |
Note if the timestamp does not display, make sure that the Time property is toggled to on.
