Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Datadog data source for Grafana Hosted Datadog Metrics

Hosted Datadog Metrics

With Grafana Labs hosted Datadog Metrics, you can forward metrics directly from your Datadog agents. To view a list of supported features, scroll down to the topic, Hosted Datadog Metrics supported features.

You must make a configuration change on the Datadog Agent to forward Datadog Agent metrics. There are two ways to do this.

Forward metrics to Datadog and Grafana Cloud

If you are evaluating hosted Datadog Metrics, use these methods so that the Datadog agents start pushing metrics to Grafana Cloud while continuing to push metrics to Datadog.

The following use cases require different configurations:

• The Datadog Agent runs as a service or on any scheduler that can use secrets, such as Kubernetes.

  Export your credentials as the environment variables GRAFANA_CLOUD_USERNAME and GRAFANA_CLOUD_APIKEY.
  After exporting your credentials, export the DD_ADDITIONAL_ENDPOINTS environment variable:

  bash Copy

  DD_ADDITIONAL_ENDPOINTS='{"https://$(GRAFANA_CLOUD_USERNAME):$(GRAFANA_CLOUD_APIKEY)@dd-cluster.grafana.net/datadog": ["grafana-labs"]}'

• The Datadog Agent runs as a system service.

  Export the environment variable in the daemon service configuration:

  bash Copy

  DD_ADDITIONAL_ENDPOINTS='{"https://<REPLACE_WITH_TENANTID>:<REPLACE_WITH_GRAFANA_APIKEY>@dd-cluster.grafana.net/datadog": ["grafana-labs"]}'

You must use grafana-labs as the API key value for the additional Grafana Labs endpoint. Do not use your Datadog API key.

Forward metrics only to Grafana Cloud

Use this configuration so that agents will only forward metrics to Grafana Cloud.

bash Copy

# The host of the Datadog intake server to send Agent data to
dd_url: https://<username>:<api-key>@dd-cluster.grafana.net/datadog
api_key: grafana-labs

# Additional endpoints to forward metrics to other backends
additional_endpoints: {}

You must use grafana-labs as the value for the api_key field. Do not use your Datadog API key.

Grafana Datadog data source configuration

Configure a new Datadog data source in your Grafana instance.

1. In your Grafana instance, click the Configuration (gear) icon.
2. Select Data Sources.
3. Click Add data source
4. Search for Datadog and fill the following fields:
   • Name: Datadog Grafana Cloud
   • API URL: https://:@dd-cluster.grafana.net/datadog
   • API Key: grafana-labs
   • App Key: grafana-labs
   • Enable Basic Auth: On
   • Basic Auth User: <Username>
   • Basic Auth Password: <Prometheus API Key>

If you wish to compare queries between the hosted Datadog Metrics and the official Datadog API, create an additional data source using the same steps but change the following fields:

Name: Datadog
API URL: https://api.datadoghq.com
API Key: <Datadog API Key>
App Key: <Datadog App Key>
Enable Basic Auth: Off

Verify that metrics are available

When all of your Datadog agents have been configured, verify that Grafana Cloud is properly receiving metrics.

1. In your Grafana instance, click the Explore (compass) icon.
2. Select Datadog Grafana Cloud from the data source dropdown menu.
3. Use the panel to query any existing metric, for example datadog.agent.running.
4. Click on Run Query to validate that it returns the expected results.

Migrate Datadog dashboards to Grafana

You can use a publicly available Grafana tool to translate your Datadog dashboard files into Grafana dashboard files.

Prerequisites

Before you migrate your dashboards, you will need to download the Grafana migration binary and migration tool release.

All the latest releases are available under the domain dl.grafana.com:

• datadog-dash-translator_latest_Darwin_arm64.tar.gz
• datadog-dash-translator_latest_Darwin_x86_64.tar.gz
• datadog-dash-translator_latest_Linux_arm64.tar.gz
• datadog-dash-translator_latest_Linux_armv6.tar.gz
• datadog-dash-translator_latest_Linux_x86_64.tar.gz
• datadog-dash-translator_latest_Windows_x86_64.tar.gz

You can also download a specific version:

bash Copy

$ agent_release_url=https://dl.grafana.com/tools/datadog-dash-translator/release
$ curl "$agent_release_url/latest_checksums.txt"
...
# choose the appropriate version and architecture from above
# for example datadog-dash-translator_0.2.0_Linux_x86_64.tar.gz
$ binary=datadog-dash-translator_0.2.0_Linux_x86_64.tar.gz
$ curl -O $binary "$agent_release_url/$binary"
$ tar -xvf $binary

Migrate a dashboard

1. Extract the binary and instructions.
   bash Copy

   $ tar -xvf datadog-dash-translator_latest_Linux_x86_64.tar.gz
   INSTRUCTIONS.md
   datadog-dash-translator

2. In the Datadog UI, find the dashboard you want to migrate.
3. Click Export dashboard JSON.
4. Locate the JSON file and run the following command:
   bash Copy

   $ ./datadog-dash-translator translate -i example-dashboard.json
   INFO[0000] Successful read of datadog file "example-dashboard.json"
   INFO[0000] Successful datadog dashboard translate
   INFO[0000] Successful grafana dashboard write to "example-dashboard.json.grafana.json"

5. Locate the new file.
   The grafana dashboard will be available in a new file, for example: example-dashboard.json.grafana.json.
6. In your Grafana instance, click the Create (+) icon and select Import
7. Click Upload JSON file and select the new file.

Hosted Datadog Metrics supported features

Datadog metrics forwarded to Grafana Cloud are converted in Prometheus metrics. Due to this conversion, we do not support all of the Datadog features yet.

Datadog Metric features

Datadog Metric types

Every metric type except DISTRIBUTION is supported. Some visualization representations will look different when a dashboard is imported to Grafana Cloud.

• COUNT: Converted into a Prometheus gauge instead of a Prometheus counter. Prometheus counters cannot handle decreases.
• RATE: Multiplied by the flush interval and then converted into a Prometheus gauge. The rate is transformed into its count equivalent.
• GAUGE: Converted to a Prometheus gauge.
• SET: Converted to a Prometheus gauge.
• HISTOGRAM: The server receives a set of GAUGE and RATE metrics that are converted.

Data Model

Translated metric names and tags (called labels in Prometheus) coming from Datadog might be slightly different when querying this data using PromQL due to character restrictions in Prometheus. This does not affect queries made using the Datadog datasource.

Other

Datadog Query coverage

Datadog features

Datadog functions

• Algorithms
  • anomalies(): no
  • forecast(): no
  • outliers(): no
• Arithmetic
  • abs(): yes
  • log2(): yes
  • log10(): no
  • cumsum(): yes
  • integral(): no
• Count
  • count_non_zero(): yes
  • count_not_null(): yes
• Exclusion
  • exclude_null(): yes
  • clamp_min(): yes
  • clamp_max(): yes
  • cutoff_min(): yes
  • cutoff_max(): yes
• Interpolation
  • fill(): no
  • default_zero(): no
• Rank
  • top(): partially, “area” and “l2norm” aggregation methods are not supported. The convenience functions (e.g. bottom10_min()) are supported
• Rate
  • per_second(): yes
  • per_minute(): yes
  • per_hour(): yes
  • dt(): no but agent sends metrics every 10s anyway.
  • diff(): yes
  • monotonic_diff(): no
  • derivative(): no
• Regression
  • robust_trend(): no
  • trend_line(): no
  • piecewise_constant(): no
• Rollup
  • rollup(): partially, see “Time Aggregation (Rollup)” in previous section
  • moving_rollup(): no
• Smoothing
  • autosmooth(): no
  • ewma_[3/5/10/20](): yes
  • median_[3/5/7/9](): yes
• Timeshift
  • timeshift(): yes
  • <TIMEPERIOD>_before(): yes

Dashboard Coverage

This section describes the Datadog features that the Datadog Dashboard Translator supports. Unknown widgets are shown as text panels in the converted dashboard.

Dashboard Types

Timeboards are supported and automatically snap to a fixed grid.

Datadog screenboards have free-form layouts and different graphs can have different time scopes, they are partially supported. The Grafana migration tool transforms a screenboard into a timeboard.

Widgets

• Alert Graph: no
• Alert Value: no
• Change: no
• Check Status: yes, status checks are converted into Prometheus metrics in the background.
• Distribution : no
• Event Stream: no
• Event Timeline: no
• Free Text: no
• Geomap: no
• Group: yes
• HeatMap: no
• Hostmap: no
• Iframe: no
• Image: no
• Log Stream: no
• Monitor Summary: no
• Network: no
• Notes and Links: no
• Query Value: partial; each widget can only have a single query.
• Scatter Plot: no
• Service Level Objective (SLO): no
• Service Map: no
• Service Summary: no
• Table: no
• Timeseries: yes
• Top List: partial; each widget can only have a single query.

Other

• Template Variables: no
• Colors: no
• Events Overlay: no