Pro tips for making the most of your Datadog metrics in Grafana with the enterprise plugin

Published: 13 Apr 2020 RSS

Hello again! We are Eldin and Christine – or, as our lovely editor has dubbed us, Regis and Kelly – jumping back in for another post.

This week, to highlight the big tent and community theme, we are going to write about how adding the Datadog plugin and visualizations alongside other sources (like Prometheus and Loki) helps provide a fuller picture of operations.

Dashboard with multiple data sources

Datadog is a popular monitoring and analytics platform that allows you to easily install an agent so you can get started with collecting metrics right away. Since we announced our Enterprise Datadog data source two years ago, the plugin has continued to evolve. We thought we were due for a blog post to show off some of our favorite features, ensuring you really can see it all in one place in no time.

Getting started

Datadog dashboard

In our example dashboard, we are visualizing some infrastructure metrics with our graph, stat, and bar gauge panels. (As of the 6.6 release, we debuted the super flexible stat panel and the beloved bar gauge got a facelift!)

To start using Datadog as a data source, install the Datadog plugin. After, add your data source to Grafana by going to Configuration > Data Sources > Add data source. Find the Datadog data source and fill out the connection details (you will need your API and App key – more information can be found here):

Add data source

Tip: For large dashboards that will be making lots of queries, it is possible to be rate-limited by the Datadog API. The Cache Interval feature helps you bypass these limits by caching unique queries. The default is 60 seconds and can be adjusted in the configuration page.

Some tips on querying

Once you’ve set up your data source, let’s head over to a new dashboard and add a panel to take a look at the query editor.

Add a panel

If you are familiar with our other query editors, you’ll notice the Datadog editor is just as easy! Select the metric, aggregation, and select one or more tags (if you want to filter results). The data source also supports all of the advanced functions with which you are familiar. Select it from the Functions dropdown and click the left/right arrows to order the functions.

One of our go-tos is the Alias By field.

Let’s say you have this metric:

Datadog dashboard

You can leverage this scoped variable $__aggr($__display_name) to replace the metric name and aggregation:

Replace metric name and aggregation

Here’s a list of available variables that can be used:

  • $__metric = replaced with metric name
  • $__display_name = replaced with metric name
  • $__expression = replaced with full metric expression
  • $__aggr = replaced with metric aggregation function (e.g. avg, max, min, sum)
  • $__scope = replaced with metric scope (e.g. region, site, env, host)

We also support ever useful regular expressions!

Tip for arithmetic queries: In the query type, switch it to Arithmetic. Use the pound sign (#) to reference a desired query. For example, #A * 2 will double the result of query A. Arithmetic between two metrics works the same way – you can perform calculations between two queries, such as #A / #B.

Arithmetic query

Templating!

Templating

There are a few options for getting values from template variables: metrics and tags. To fetch the list of available metrics, specify the star symbol (*) in the Query field (or to return all tags, input the value tag or scope).

To return tags for a specified tag group, use one of the following default category values:

  • host
  • device
  • env
  • region
  • site
  • status
  • version

Custom tag groups and filtered results are possible by using the Regex field.

One last thing

Our last tip is to leverage ad-hoc filters. This variable will apply to all the Datadog queries using the variable in a dashboard, so you can use it like a quick filter. An ad-hoc variable for Datadog fetches all the key-value pairs from tags (for example, region:east, region:west) and uses them as query tags. To create this variable, select the Ad-hoc filter type and choose your Datadog data source. You can set any name for this variable.

That’s all we have time for today, but please tweet at us to let us know the next plugin you’d like us to cover. Happy dashboarding and until next time! Kelly and Regis out!

Kelly and Regis out!

Related Posts

Here’s everything you need to know about the latest addition to our burgeoning assemblage of Enterprise plugins: the ServiceNow integration with Grafana.
Wondering how Grafana Labs customers are benefiting from using Grafana and hosted Prometheus/Cortex? Here are four success stories.
I’ve talked to hundreds of customers about their observability challenges. Here’s what I heard about why they chose Grafana Enterprise.

Related Case Studies

How 84.51° centralized its metrics with Grafana Enterprise

Grafana Enterprise helps us tell a ‘holistic story in a one-stop shop,’ says Lead Service Manager Erin O’Brien.

"We have so much data, but how do you take that data and tell the story? Grafana is our storyteller."
– Erin O’Brien, Lead Service Manager, 84.51°

After trying to DIY, Wix embraces Grafana Cloud

Metrics is an important part of Wix’s culture, so Grafana Cloud was chosen to monitor mission-critical systems.

"It doesn’t make sense, price-to-performance, to do it ourselves, so we were looking for a fully-managed solution from a team that had experience running monitoring at scale."
– Alex Ulstein, Head of Monitoring, Wix

DigitalOcean gains new insight with Grafana visualizations

The company relies on Grafana to be the consolidated data visualization and dashboard solution for sharing data.

"Grafana produces beautiful graphs we can send to our customers, works with our Chef deployment process, and is all hosted in-house."
– David Byrd, Product Manager, DigitalOcean