Build a data source backend plugin

Introduction

Grafana supports a wide range of data sources, including Prometheus, MySQL, and even Datadog. There’s a good chance you can already visualize metrics from the systems you have set up. In some cases, though, you already have an in-house metrics solution that you’d like to add to your Grafana dashboards. This tutorial teaches you to build a support for your data source.

For more information about backend plugins, refer to the documentation on Backend plugins.

In this tutorial, you’ll:

  • Build a backend for your data source
  • Implement a health check for your data source
  • Enable Grafana Alerting for your data source

Prerequisites

  • Knowledge about how data sources are implemented in the frontend.
  • Grafana 7.0
  • Go 1.14+
  • Mage
  • NodeJS
  • yarn

Set up your environment

Before you can get started building plugins, you need to set up your environment for plugin development.

To discover plugins, Grafana scans a plugin directory, the location of which depends on your operating system.

  1. Create a directory called grafana-plugins in your preferred workspace.

  2. Find the plugins property in the Grafana configuration file and set the plugins property to the path of your grafana-plugins directory. Refer to the Grafana configuration documentation for more information.

    [paths]
    plugins = "/path/to/grafana-plugins"
    
  3. Restart Grafana if it’s already running, to load the new configuration.

Alternative method: Docker

If you don’t want to install Grafana on your local machine, you can use Docker.

To set up Grafana for plugin development using Docker, run the following command:

docker run -d -p 3000:3000 -v "$(pwd)"/grafana-plugins:/var/lib/grafana/plugins --name=grafana grafana/grafana:7.0.0

Since Grafana only loads plugins on start-up, you need to restart the container whenever you add or remove a plugin.

docker restart grafana

Create a new plugin

To build a backend for your data source plugin, Grafana requires a binary that it can execute when it loads the plugin during start-up. In this guide, we will build a binary using the Grafana plugin SDK for Go.

The easiest way to get started is to clone one of our test data datasources. Navigate to the plugin folder that you configured in step 1 and type:

npx @grafana/toolkit@next plugin:create my-plugin

Select Backend Datasource Plugin and follow the rest of the steps in the plugin scaffolding command.

cd my-plugin

Install frontend dependencies and build frontend parts of the plugin to dist directory:

yarn install
yarn build

Run the following to update Grafana plugin SDK for Go dependency to the latest minor version:

go get -u github.com/grafana/grafana-plugin-sdk-go

Build backend plugin binaries for Linux, Windows and Darwin to dist directory:

mage -v

Now, let’s verify that the plugin you’ve built so far can be used in Grafana when creating a new data source:

  1. Restart your Grafana instance.
  2. Open Grafana in your web browser.
  3. Navigate via the side-menu to Configuration -> Data Sources.
  4. Click Add data source.
  5. Find your newly created plugin and select it.
  6. Enter a name and then click Save & Test (ignore any errors reported for now).

You now have a new data source instance of your plugin that is ready to use in a dashboard:

  1. Navigate via the side-menu to Create -> Dashboard.
  2. Click Add new panel.
  3. In the query tab, select the data source you just created.
  4. A line graph is rendered with one series consisting of two data points.
  5. Save the dashboard.

Anatomy of a backend plugin

The folders and files used to build the backend for the data source are:

file/folder description
Magefile.go It’s not a requirement to use mage build files, but we strongly recommend using it so that you can use the build targets provided by the plugin SDK.
/go.mod Go modules dependencies, reference
/src/plugin.json A JSON file describing the backend plugin
/pkg/main.go Starting point of the plugin binary.

plugin.json

The plugin.json file is required for all plugins. When building a backend plugin these properties are important:

property description
backend Should be set to true for backend plugins. This tells Grafana that it should start a binary when loading the plugin.
executable This is the name of the executable that Grafana expects to start, see plugin.json reference for details.
alerting Should be set to true if your backend datasource supports alerting.

In the next step we will look at the query endpoint!

Implement data queries

We begin by opening the file /pkg/sample-plugin.go. In this file you will see the SampleDatasource struct which implements the backend.QueryDataHandler interface. The QueryData method on this struct is where the data fetching happens for a data source plugin.

Each request contains multiple queries to reduce traffic between Grafana and plugins. So you need to loop over the slice of queries, process each query, and then return the results of all queries.

In the tutorial we have extracted a method named query to take care of each query model. Since each plugin has their own unique query model, Grafana sends it to the backend plugin as JSON. Therefore the plugin needs to Unmarshal the query model into something easier to work with.

As you can see the sample only returns static numbers. Try to extend the plugin to return other types of data.

You can read more about how to build data frames in our docs.

Add support for health checks

Implementing the health check handler allows Grafana to verify that a data source has been configured correctly.

When editing a data source in Grafana’s UI, you can Save & Test to verify that it works as expected.

In this sample data source, there is a 50% chance that the health check will be successful. Make sure to return appropriate error messages to the users, informing them about what is misconfigured in the data source.

Open /pkg/sample-plugin.go. In this file you’ll see that the SampleDatasource struct also implements the backend.CheckHealthHandler interface. Navigate to the CheckHealth method to see how the health check for this sample plugin is implemented.

Enable Grafana Alerting

  1. Open src/plugin.json.
  2. Add the top level backend property with a value of true to specify that your plugin supports Grafana Alerting, e.g.
    {
      ...
      "backend": true,
      "executable": "gpx_simple_datasource_backend",
      "alerting": true,
      "info": {
      ...
    }
    
  3. Rebuild frontend parts of the plugin to dist directory:
yarn build
  1. Restart your Grafana instance.
  2. Open Grafana in your web browser.
  3. Open the dashboard you created earlier in the Create a new plugin step.
  4. Edit the existing panel.
  5. Click on the Alert tab.
  6. Click on Create Alert button.
  7. Edit condition and specify IS ABOVE 10. Change Evaluate every to 10s and clear the For field to make the alert rule evaluate quickly.
  8. Save the dashboard.
  9. After some time the alert rule evaluates and transitions into Alerting state.

Congratulations

Congratulations, you made it to the end of this tutorial!