Instrument a Python application
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Monitor applicationsbreadcrumb arrow Application Observabilitybreadcrumb arrow Instrument an applicationbreadcrumb arrow Python
Grafana Cloud

Instrument a Python application

For most use cases Grafana Labs recommends Beyla, eBPF network-level auto-instrumentation, which is easy to set up and supports all languages and frameworks.

If you need process-level telemetry for Python, follow this documentation to set up the upstream OpenTelemetry SDK for Python for Application Observability.

Install the SDK

Before you begin ensure you have a Python 3 development environment and a Python application to instrument.

Run the following command in the project folder:

sh 
pip install "opentelemetry-distro[otlp]"
opentelemetry-bootstrap -a install

Instrument your application

Export the following environment variables before starting your application to enable auto-instrumentation:

sh 
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
export OTEL_LOGS_EXPORTER=otlp

Global Interpreter Lock

The OpenTelemetry Python SDK uses a Global Interpreter Lock (GIL) which can cause performance issues with web servers that spawn multiple processes to serve requests in parallel.

To fix this, register a post fork hook that is called after each worker process is forked.

The following code sample includes a post fork hook for Gunicorn:

python 
import logging
from uuid import uuid4

from opentelemetry import metrics, trace
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
    OTLPLogExporter,
)
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import (
    OTLPMetricExporter,
)
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
    OTLPSpanExporter,
)

# support for logs is currently experimental
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.resources import SERVICE_INSTANCE_ID
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# your gunicorn config here
# bind = "127.0.0.1:8000"

collector_endpoint = "http://localhost:4317"

def post_fork(server, worker):
    server.log.info("Worker spawned (pid: %s)", worker.pid)

    resource = Resource.create(
        attributes={
            # each worker needs a unique service.instance.id to distinguish the created metrics in prometheus
            SERVICE_INSTANCE_ID: str(uuid4()),
            "worker": worker.pid,
        }
    )

    tracer_provider = TracerProvider(resource=resource)
    tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint=collector_endpoint)))
    trace.set_tracer_provider(tracer_provider)

    metrics.set_meter_provider(
        MeterProvider(
            resource=resource,
            metric_readers=[(PeriodicExportingMetricReader(
                OTLPMetricExporter(endpoint=collector_endpoint)
            ))],
        )
    )

    logger_provider = LoggerProvider(resource=resource)
    logger_provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter(endpoint=collector_endpoint)))
    logging.getLogger().addHandler(LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider))

Run the Gunicorn script using the following command gunicorn app -c gunicorn.conf.py.

Note

To register a post for hook for other web servers, such as uWSGI, add the @postfork decorator to your hook function.

Test your instrumentation

To test if you’ve successfully instrumented your application, run your application, generate some traffic, and you should see metrics and logs outputted to the console.

sh 
opentelemetry-instrument flask run -p 8080

Example application

See the Rolldice service for a complete example setup.

Next steps

  1. Create a free Grafana Cloud account.
  2. For a local development and testing, send data to the Grafana Cloud OTLP endpoint.
  3. For production, set up an OpenTelemetry Collector.
  4. Observe your services in Application Observability.

Resources

  1. OpenTelemetry Distro OpenTelemetry documentation
  2. opentelemetry-distro on pypi
  3. opentelemetry-python-contrib on GitHub