Datadog Metrics

If you have a Grafana Cloud Advanced account with access to Enterprise plugins, you can forward metrics with Grafana Labs Datadog Metrics directly from your Datadog Agents. To learn more about the Grafana Cloud Enterprise stack, contact us.

To view a list of supported features, see Datadog metrics supported features.

The following configuration refers to <dd-cluster>.grafana.net in several places. The value of <dd-cluster> depends on your specific Grafana Cloud stack. To find out the value for your Grafana Cloud stack, go to the Prometheus metrics details in the desired stack, take the domain from the URL and replace prometheus- with dd-.

For example, if your Prometheus URL is https://prometheus-us-central1.grafana.net/api/prom, your Hosted Datadog Metrics API can be found on https://dd-us-central1.grafana.net/datadog.

Forwarding metrics

To forward metrics, you need an API key with the MetricsPublisher role.

You must make a configuration change on the Datadog Agent to forward Datadog metrics. There are two ways to do this.

Forward metrics to Datadog and Grafana Cloud

If you are evaluating Datadog Metrics, use these methods so that the Agents handle the metric pushes to different endpoints in different, isolated queues. This ensures that problems that occur in one or more receivers will not affect other receivers.

The following use cases require different configurations:

• The Datadog Agent runs as a service or on any scheduler that can use secrets, such as Kubernetes.

  Export your credentials as the environment variables GRAFANA_CLOUD_USERNAME and GRAFANA_CLOUD_APIKEY.
  After exporting your credentials, export the DD_ADDITIONAL_ENDPOINTS environment variable:

  DD_ADDITIONAL_ENDPOINTS='{"https://$(GRAFANA_CLOUD_USERNAME):$(GRAFANA_CLOUD_APIKEY)@<dd-cluster>.grafana.net/datadog": ["grafana-labs"]}'
  

• The Datadog Agent runs as a system service.

  Export the environment variable in the daemon service configuration:

  DD_ADDITIONAL_ENDPOINTS='{"https://<REPLACE_WITH_TENANTID>:<REPLACE_WITH_GRAFANA_APIKEY>@<dd-cluster>.grafana.net/datadog": ["grafana-labs"]}'
  

You must use grafana-labs as the API key value for the additional Grafana Labs endpoint. Do not use your Datadog API key.

This is similar to setting the additional_endpoints key in datadog.yaml, however you can’t paste your API key directly into the YAML file because Datadog Agent will convert it to lowercase as part of the config parsing. Setting it through the DD_ADDITIONAL_ENDPOINTS environment variable does not go through that lowercase conversion.

Forward metrics only to Grafana Cloud

If you are using Grafana Datadog Metrics in a production environment, use this configuration so that Agents will only forward metrics to Grafana Cloud.

# The host of the Datadog intake server to send Agent data to
dd_url: https://<username>:<api-key>@<dd-cluster>.grafana.net/datadog
api_key: grafana-labs

# Additional endpoints to forward metrics to other backends
additional_endpoints: {}

You must use grafana-labs as the value for the api_key field. Do not use your Datadog API key.

Grafana Datadog data source configuration

Configure a new Datadog data source in your Grafana instance.

1. In your Grafana instance, click the Configuration (gear) icon.
2. Select Data Sources.
3. Click Add data source.
4. Scroll down to Enterprise plugins and install DataDog.
5. Complete the following fields:
   • Name: Datadog Grafana Cloud
   • API URL: https://<dd-cluster>.grafana.net/datadog
   • API Key: grafana-labs
   • App Key: grafana-labs
   • Enable Basic Auth: On
   • Basic Auth User: <Username>
   • Basic Auth Password: <Prometheus_API_Key>
     • Your API Key needs to have read permissions (e.g. the Viewer or Editor role)

To compare queries between your data source and the Datadog official API, create an additional data source using the same steps, but change following fields:

• Name: Datadog
• API URL: https://api.datadoghq.com
• API Key: <Datadog_API_Key>
• App Key: <Datadog_App_Key>
• Enable Basic Auth: Off

Verify that metrics are available

When all of your Datadog Agents have been configured, verify that Grafana Cloud is properly receiving metrics.

1. In your Grafana instance, click the Explore (compass) icon.
2. Select Datadog Grafana Cloud from the data source dropdown menu.
3. Use the panel to query any existing metric, for example datadog.Agent.running.
4. Click on Run Query to validate that it returns the expected results.

Migrate Datadog dashboards to Grafana

You can use a publicly available Grafana tool to translate your Datadog dashboard files into Grafana dashboard files.

Prerequisites

Before you migrate your dashboards, you will need to download the Grafana migration binary and migration tool release.

Fetch the binary from the public URL
Releases are available under the domain dl.grafana.com.

$ agent_release_url=https://dl.grafana.com/tools/datadog-dash-translator/release
$ curl "$agent_release_url/latest_checksums.txt"
...
# choose the appropriate version and architecture from above
# for example datadog-dash-translator_0.1.0_Linux_x86_64.tar.gz
$ binary=datadog-dash-translator_0.1.0_Linux_x86_64.tar.gz
$ curl -O $binary "$agent_release_url/$binary"
$ tar -xvf $binary

Find a release
Access the list of releases using the CLI command, gsutil.

• If you have Docker installed run the following commands:

$ BUCKET=gs://grafana-downloads/tools/datadog-dash-translator/release
$ docker run -it google/cloud-sdk:alpine gsutil ls $BUCKET

• If gsutil is available locally run:

$ BUCKET=gs://grafana-downloads/tools/datadog-dash-translator/release
$ gsutil ls $BUCKET

Migrate a dashboard

1. Extract the binary and instructions.

   $ tar -xvf datadog-dash-translator_0.1.0_Linux_x86_64.tar.gz
   INSTRUCTIONS.md
   datadog-dash-translator
   

2. In the Datadog UI, find the dashboard you want to migrate.
3. Click Export dashboard JSON.
4. Locate the JSON file and run the following command:

   $ ./datadog-dash-translator translate -i example-dashboard.json
   INFO[0000] Successful read of datadog file "example-dashboard.json"
   INFO[0000] Successful datadog dashboard translate
   INFO[0000] Successful grafana dashboard write to "example-dashboard.json.grafana.json"
   

5. Locate the new file. The grafana dashboard will be available in a new file, for example: example-dashboard.json.grafana.json.
6. In your Grafana instance, click the Create (+) icon and select Import
7. Click Upload JSON file and select the new file.