This is documentation for the next version of Grafana Alloy Documentation. For the latest stable release, go to the latest version.

Documentation Grafana Alloy Reference Components prometheus prometheus.exporter.databricks

prometheus.exporter.databricks

The prometheus.exporter.databricks component embeds the databricks-prometheus-exporter to collect billing, jobs, pipelines, and SQL warehouse metrics from Databricks System Tables.

You can specify multiple prometheus.exporter.databricks components by giving them different labels.

Prerequisites

Before you configure this component, make sure you have the following:

• Databricks workspace: Enable Unity Catalog and System Tables.
• Service principal: Configure OAuth2 M2M authentication.
• SQL warehouse: Query System Tables through an SQL warehouse. A serverless warehouse costs the least.

Refer to the Databricks documentation for detailed OAuth2 M2M setup instructions.

Usage

Alloy Copy

prometheus.exporter.databricks "<LABEL>" {
  server_hostname     = "<DATABRICKS_SERVER_HOSTNAME>"
  warehouse_http_path = "<DATABRICKS_WAREHOUSE_HTTP_PATH>"
  client_id           = "<DATABRICKS_CLIENT_ID>"
  client_secret       = "<DATABRICKS_CLIENT_SECRET>"
}

Arguments

You can use the following arguments with prometheus.exporter.databricks:

The component sets the instance label on its exported targets to the value of server_hostname.

Lookback windows

The exporter queries Databricks System Tables using SQL with sliding time windows. Each scrape collects data from now - lookback to now:

• billing_lookback: Queries system.billing.usage for DBU consumption and cost estimates. Databricks billing data typically has 24-48 hour lag.
• jobs_lookback: Queries system.lakeflow.job_run_timeline for job run counts, durations, and status.
• pipelines_lookback: Queries system.lakeflow.pipeline_event_log for DLT pipeline metrics.
• queries_lookback: Queries system.query.history for SQL warehouse query metrics.

The lookback window should be at least 2x the scrape_interval to ensure data continuity between scrapes. For example, with a 10-minute scrape interval, use at least 20 minutes of lookback.

Blocks

The prometheus.exporter.databricks component doesn’t support any blocks. You can configure this component with arguments.

Exported fields

Component health

prometheus.exporter.databricks is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

prometheus.exporter.databricks doesn’t expose any component-specific debug information.

Debug metrics

prometheus.exporter.databricks doesn’t expose any component-specific debug metrics.

Performance recommendations

• scrape_interval: Use 10-30 minutes. The exporter queries Databricks System Tables which can be slow. Increase the scrape_interval to reduce your SQL Warehouse costs.
• scrape_timeout: Must be less than scrape_interval. The exporter typically takes 90-120 seconds per scrape depending on data volume.
• Lookback vs interval: The lookback windows should be at least 2x the scrape interval. The defaults, 3h for jobs and pipelines, and 2h for queries, work well with 10-30 minute scrape intervals.

High cardinality warning

The collect_task_retries flag adds task-level retry metrics which can significantly increase cardinality for workspaces with many jobs. Only enable this feature if you really need it.

Example

The following example uses a prometheus.scrape component to collect metrics from prometheus.exporter.databricks:

Alloy Copy

prometheus.exporter.databricks "example" {
  server_hostname     = "dbc-abc123-def456.cloud.databricks.com"
  warehouse_http_path = "/sql/1.0/warehouses/xyz789"
  client_id           = "my-service-principal-id"
  client_secret       = "my-service-principal-secret"
}

// Configure a prometheus.scrape component to collect databricks metrics.
prometheus.scrape "demo" {
  targets         = prometheus.exporter.databricks.example.targets
  forward_to      = [prometheus.remote_write.demo.receiver]
  scrape_interval = "10m"
  scrape_timeout  = "9m"
}

prometheus.remote_write "demo" {
  endpoint {
    url = "<PROMETHEUS_REMOTE_WRITE_URL>"

    basic_auth {
      username = "<USERNAME>"
      password = "<PASSWORD>"
    }
  }
}

Replace the following:

• <PROMETHEUS_REMOTE_WRITE_URL>: The URL of the Prometheus remote_write compatible server to send metrics to.
• <USERNAME>: The username to use for authentication to the remote_write API.
• <PASSWORD>: The password to use for authentication to the remote_write API.

Compatible components

prometheus.exporter.databricks has exports that can be consumed by the following components:

• Components that consume Targets