Multitenant
Companies with multiple SRE teams are starting to have a central platform engineering team providing the tooling for the individual product teams to manage part of their observability tools. In that scenario, it’s helpful to allow teams to use an OpenTelemetry Collector with their own configuration, using the receivers and processors they need, while still having a central layer of collectors, deciding where the telemetry data is sent. Different teams might have different accounts at a backend service, or use different data stores altogether. In that case, the centralized OpenTelemetry Collector needs to understand which tenant data goes to which backend. In addition, the platform engineering team might also want to collect usage metrics, so that product teams are charged back based on usage.
Example of a collector configuration for the tenant acme:
extensions:
receivers:
otlp:
protocols:
grpc:
processors:
exporters:
otlp:
endpoint: my-otelcol.observability.svc.cluster.local:4317
headers:
tenant: acme
service:
extensions: []
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [otlp]
Example of a collector configuration for the tenant ecorp:
extensions:
receivers:
otlp:
protocols:
grpc:
processors:
exporters:
otlp:
endpoint: my-otelcol.observability.svc.cluster.local:4317
headers:
tenant: ecorp
service:
extensions: []
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [otlp]
Example of a central collector configuration:
extensions:
basicauth/fallback:
client_auth:
username: "${FALLBACK_USER_ID}"
password: "${FALLBACK_TOKEN}"
basicauth/acme:
client_auth:
username: "${ACME_USER_ID}"
password: "${ACME_TOKEN}"
basicauth/ecorp:
client_auth:
username: "${ECORP_USER_ID}"
password: "${ECORP_TOKEN}"
receivers:
otlp:
protocols:
grpc:
processors:
routing:
from_attribute: tenant
default_exporters: otlphttp/fallback
table:
- value: acme
exporters: [otlphttp/acme]
- value: ecorp
exporters: [otlphttp/ecorp]
exporters:
otlphttp/fallback:
endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp
auth:
authenticator: basicauth/fallback
otlphttp/acme:
endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp
auth:
authenticator: basicauth/acme
otlphttp/ecorp:
endpoint: https://otlp-gateway-prod-us-central-0.grafana.net/otlp
auth:
authenticator: basicauth/ecorp
service:
extensions: [basicauth/fallback, basicauth/acme, basicauth/ecorp]
pipelines:
traces:
receivers: [otlp]
processors: [routing]
exporters: [otlp/fallback, otlp/acme, otlp/ecorp]