Visualize Databricks in Grafana: write custom SQL queries, build interactive dashboards, and more
As part of our big tent philosophy at Grafana Labs, we think you should be able to dig into your data and find meaningful insights — wherever that data happens to live.
For many of our users, that data lives in Databricks, the open analytics platform for building, deploying, sharing, and maintaining enterprise-grade data, analytics, and AI solutions at scale.
In this post, we’ll take a closer look at how you can use the Databricks Enterprise data source for Grafana to quickly and easily visualize your Databricks data using Grafana dashboards. You can also learn more about the data source, and see a demo, in the YouTube video below.
A quick overview of Grafana Enterprise data sources
Grafana Enterprise data sources allow you to connect to external systems and popular proprietary platforms from within Grafana, so you can visualize data from a wide range of sources using the Grafana dashboards you already know and love. While many Grafana data sources are built by our partners or community members, Enterprise data sources are built and maintained by Grafana Labs, and they are only available in Grafana Enterprise and across all Grafana Cloud tiers, including our generous Cloud free tier.
Databricks is one of more than 30 Enterprise data sources currently in our catalog, alongside ServiceNow, Splunk, Snowflake, Datadog, Dynatrace, and AppDynamics.
To learn more about Grafana Enterprise data sources, check out our recent guide to Enterprise data sources.
Why visualize Databricks in Grafana
With the Databricks Enterprise data source, you can easily query and visualize Databricks data in Grafana Enterprise or Grafana Cloud. For example, you can write a custom SQL query to access system metrics and billing data from Databricks, and then track things like compute usage and costs directly within your Grafana dashboards.
Other key features of the Databricks data source include:
- SQL editor: Format and color-code your SQL statements, with support for autocompletion.
- Time series support: Visualize time series data by including a datetime field type in your query. This field will be used as the timestamp.
- Macros: Use
Grafana macros (e.g.,
$__timeFilteror$__interval_long) to simplify time-based queries. - Variables and templates: Integrate Grafana variables into your Databricks queries to make your dashboards more interactive and dynamic.
We’re always working to add new features to the Databricks data source and make it even easier to use. Recent updates include a schema-aware SQL query builder to help you build faster queries, and support for Azure On-behalf-of authentication, which is recommended when you need to authenticate with both Azure Databricks and other Azure resources simultaneously.
Grafana Labs at the Databricks Data + AI Summit
Grafana Labs is excited to be a sponsor at the Databricks Data + AI Summit, which runs June 9-12 in San Francisco.
Come visit us at booth E202 to learn about new observability features, see demos on controlling costs/maximizing value, and hear how we’ve incorporated AI/ML to enhance your observability practice. You’ll also get to learn about the latest features in open source observability, meet the technical pros behind our products, and, of course, grab some swag.
Learn more
To learn more about the Databricks Enterprise data source, please check out our technical docs. We also have several resources you can use to learn more about Grafana Enterprise data sources, in general, including our documentation, plugin catalog, and this webinar.
The easiest way to get started with our Enterprise data source plugins is in Grafana Cloud. We have a generous forever-free tier that includes access to Enterprise data sources for 3 active users. If you haven’t already, sign up for free today!
