Hubble datasource plugin for Grafana

Hubble datasource plugin allows querying data produced by Hubble - the observability layer of Cilium - from Grafana.

Some of the features work with Cilium OSS, others require Isovalent Cilium Enterprise - those are marked with an info box.

Configuration

There are three underlying data stores that can be configured in Hubble data source:

• Timescape - an observability and analytics platform for eBPF collected Hubble flow data and Tetragon process events which allows querying historical data shipped as part of Isovalent Cilium Enterprise. It's used in Flows and Process Ancestry queries.
• Prometheus - must store Hubble metrics, used in Service Map query
• Tempo - used in Flows query

Queries

Hubble datasource plugin supports three query types: Service Map, Flows, and Process Ancestry. Queries support dashboard variables, with possible limitations.

Service Map

The Service Map query renders an interactive service map from Hubble HTTP metrics. It's designed to be visualized as a node graph. The graph nodes contain dropdowns with links to detailed Prometheus queries.

For Service Map to work correctly, Hubble HTTP metrics have to be enabled with correct labels: at least source and destination. Filters require additional labels, e.g. destination_namespace. Refer to the Hubble metrics documentation for configuration details. This also requires Hubble Layer 7 visibility to be configured for your HTTP services, in order to the HTTP metrics to be generated. Below is an example of Helm values for Cilium with Hubble metrics enabled:

hubble:
  metrics:
    enabled:
      - "httpV2:sourceContext=workload-name|pod-name|reserved-identity;destinationContext=workload-name|pod-name|reserved-identity;labelsContext=source_namespace,destination_namespace,traffic_direction"
      - "kafka:sourceContext=workload-name|pod-name|reserved-identity;destinationContext=workload-name|pod-name|reserved-identity;labelsContext=source_namespace,destination_namespace,traffic_direction"
    enableOpenMetrics: true
    serviceMonitor:
      enabled: true
    dashboards:
      enabled: true
      namespace: monitoring

Flows

The Flows query requires a connection to a Timescape server storing Hubble flows. It retrieves raw network flows for the selected time range and filters.

If applications propagate trace context following the W3C standard, then Hubble flows will contain trace ID. Providing a Tempo datasource is linked in Hubble datasource settings, trace IDs in Flows query results will link to Tempo queries.

Process Ancestry

The Process Ancestry query requires a connection to a Timescape server storing Tetragon process events. It renders a process ancestry tree for the selected pod. It's designed to be used with Process Ancestry panel plugin.

Limitations

• At the moment Timescape connection doesn't support TLS and authentication. That means Timescape has to be deployed without TLS or Hubble RBAC enabled. Support for both is planned before the first stable (production-ready) release.
• Some of the filters are multi-select and multi-value dashboard variables, but not all. Expect breaking changes to the filters before the first stable (production-ready) release.