How to convert a frontend data source plugin into a backend plugin
This guide shows you how to convert an existing frontend-only data source plugin into a backend plugin.
To convert the frontend data source, we recommend scaffolding a new backend data source plugin using npx @grafana/create-plugin@latest. Use the following instructions to extend this foundation to copy functionality from your original plugin.
Why
There are multiple features available only in backend plugins, such as Grafana Alerting or Recorded queries. See the use cases for implementing a backend plugin in the backend plugins introduction.
Before you begin
Before you dive into the details, you should familiarize yourself with the process of creating a backend data source plugin. If you haven't done this before, you can follow our tutorial for building a backend plugin.
Key concepts
Before going into specific conversion advice, it's important to understand the main components of a data source and how these differ between frontend and backend plugins.
Frontend DataSource class
Data source plugins implement a new DataSourcePlugin. This class takes as a parameter a DataSource class, which for frontend data sources extends DataSourceApi, and for backend data sources extends DataSourceWithBackend. Because the DatasourceWithBackend class already implements most of the required methods, you can migrate to it to significantly simplify your code.
Data source plugins require two components: a query editor and a config editor.
Examples:
Query and config editor
These two frontend components do not need to be changed when converting a frontend data source to a backend data source. However, if you add a backend component to a data source you can request resources from it. Resources are additional endpoints that the plugin exposes and can be used to populate or validate the query or config editor. Learn about this in the resource requests section.
Plugin structure comparison
The following folders illustrate the new components that are introduced when you add a backend to the plugin:
myorg-myplugin-datasource/
├── .config/
├── .eslintrc
├── .github
│ └── workflows
├── .gitignore
├── .nvmrc
├── .prettierrc.js
├── CHANGELOG.md
├── LICENSE
├── Magefile.go # Build definition for backend executable
├── README.md
│ └── integration
├── docker-compose.yaml
├── go.mod # Dependencies
├── go.sum # Checksums
├── jest-setup.js
├── jest.config.js
├── node_modules
├── package.json
├── pkg
│ ├── main.go # Entry point for backend
│ └── plugin # Other plugin packages
├── playwright.config.ts
├── src
│ ├── README.md
│ ├── components
│ ├── datasource.ts
│ ├── img
│ ├── module.ts
│ ├── plugin.json # Modified to include backend=true and executable=<name-of-built-binary>
│ └── types.ts
├── tsconfig.json
└── tests
Convert frontend to backend functions
Most plugins only need to implement three methods to be fully functional: a function to run queries, a function to test the data source connection, and any additional GET requests to retrieve different resources (used to populate the query editor or config editor). All three methods usually share the same authentication mechanism against the target data source.
Now let's discuss how to move the authentication logic from the frontend to the backend.
Authentication
Grafana data sources typically include two types of data: jsonData and secureJsonData. The former is used to store non-sensitive information, while the latter is used to store sensitive information like passwords or API keys.
Both frontend and backend types use the same JSON data to authenticate against the target data source. The main difference is that frontend data sources should read and use credentials for every request while backend data sources should share the same authenticated client between requests.
In a frontend-only data source, any request that requires authentication needs to go through the plugin proxy. You need to define a routes object within the plugin.json file and specify there the URL and credentials to use for each request. For example, you can authenticate a request to a given URL by setting an Authorization header with the SecureJsonData credentials:
"routes": [
{
"path": "example",
"url": "https://api.example.com",
"headers": [
{
"name": "Authorization",
"content": "Bearer {{ .SecureJsonData.apiToken }}"
}
]
}
]
To use this route, the frontend data source should call the fetch method from the DataSourceApi class. This method proxies the request and adds the Authorization header:
import { getBackendSrv } from '@grafana/runtime';
const routePath = '/example';
const res = getBackendSrv().datasourceRequest({
url: this.url + routePath + '/v1/users',
method: 'GET',
});
// Handle response
In a backend data source, you should move the authentication logic to the Datasource constructor. This method is called when the data source is created and should be used to create the authenticated client. Store this client in the Datasource instance and use it for every request. For example:
package plugin
import (
...
"github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
...
)
func NewDatasource(ctx context.Context, settings backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
opts, err := settings.HTTPClientOptions(ctx)
if err != nil {
return nil, fmt.Errorf("http client options: %w", err)
}
opts.Header.Add("Authorization", "Bearer " + settings.DecryptedSecureJSONData["token"])
cli, err := httpclient.New(opts)
if err != nil {
return nil, fmt.Errorf("httpclient new: %w", err)
}
return &Datasource{
httpClient: cl,
}, nil
}
// In any other method
res, err := d.httpClient.Get("https://api.example.com/v1/users")
// Handle response
The same principle applies to any other authentication mechanism. For example, SQL-based data sources should use the Datasource constructor to create a connection to the database and store it in the Datasource instance.
You can refer to this example and get more information about plugin authentication.
Health check
Once you move authentication logic to the backend, you can do a health check in the backend.
You need to delete the frontend implementation testDatasource in your Datasource class in the frontend to use the health check in the backend.
In this frontend example, the health check makes an API request to https://api.example.com (as defined in the routes field in plugin.json) and returns an error if the request fails:
import { getBackendSrv } from '@grafana/runtime';
const routePath = '/example';
export class MyDatasource extends DataSourceApi<MyQuery, MyDataSourceJsonData> {
...
async testDatasource() {
try {
await getBackendSrv().datasourceRequest({
url: this.url + routePath + '/v1/users',
method: 'GET',
});
return {
status: 'success',
message: 'Health check passed.',
};
} catch (error) {
return { status: 'error', message: error.message };
}
}
}
In the case of a backend data source, the Datasource struct should implement the CheckHealth method. This method returns an error if the data source is not healthy. For example:
func NewDatasource(ctx context.Context, settings backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
opts, err := settings.HTTPClientOptions(ctx)
if err != nil {
return nil, fmt.Errorf("http client options: %w", err)
}
cl, err := httpclient.New(opts)
if err != nil {
return nil, fmt.Errorf("httpclient new: %w", err)
}
return &Datasource{
settings: settings,
httpClient: cl,
}, nil
}
func (d *Datasource) CheckHealth(ctx context.Context, _ *backend.CheckHealthRequest) (*backend.CheckHealthResult, error) {
resp, err := d.httpClient.Get(d.settings.URL + "/v1/users")
if err != nil {
// Log the error here
return &backend.CheckHealthResult{
Status: backend.HealthStatusError,
Message: "request error",
}, nil
}
if resp.StatusCode != http.StatusOK {
return &backend.CheckHealthResult{
Status: backend.HealthStatusError,
Message: fmt.Sprintf("got response code %d", resp.StatusCode),
}, nil
}
return &backend.CheckHealthResult{
Status: backend.HealthStatusOk,
Message: "Data source is working",
}, nil
}
This example covers an HTTP-only data source. So, if your data source requires a database connection, you can use the Go client for the database and execute a simple query like SELECT 1 or a ping function.
Query
The next step is to move the query logic. This will significantly vary depending on how the plugin queries the data source and transforms the response into frames. In this guide, you'll see how to migrate a simple example.
Our data source is returning a JSON object with a list of datapoints when hitting the endpoint /metrics. The frontend query method transforms those datapoints into frames:
export class DataSource extends DataSourceApi<MyQuery, MyDataSourceOptions> {
async query(options: DataQueryRequest<MyQuery>): Promise<DataQueryResponse> {
const response = await lastValueFrom(
getBackendSrv().fetch<DataSourceResponse>({
url: `${this.url}/metrics`,
method: 'GET',
})
);
const df: DataFrame = {
length: response.data.datapoints.length,
refId: options.targets[0].refId,
fields: [
{ name: 'Time', values: [], type: FieldType.time, config: {} },
{
name: 'Value',
values: [],
type: FieldType.number,
config: {},
},
],
};
response.data.datapoints.forEach((datapoint: any) => {
df.fields[0].values.push(datapoint.time);
df.fields[1].values.push(datapoint.value);
});
return { data: [df] };
}
}
Now let's see how this can be translated to the backend. The Datasource instance should implement the QueryData method. This method should return a list of frames.
As with the health check, you need to delete the frontend implementation query in your Datasource class in the frontend.
The following example shows the preceding method:
func (d *Datasource) QueryData(ctx context.Context, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error) {
res, err := d.httpClient.Get(d.settings.URL + "/metrics")
// Handle errors (omitted)
// Decode response
var body struct {
DataPoints []apiDataPoint `json:"datapoints"`
}
if err := json.NewDecoder(httpResp.Body).Decode(&body); err != nil {
return backend.DataResponse{}, fmt.Errorf("%w: decode: %s", errRemoteRequest, err)
}
// Create slice of values for time and values.
times := make([]time.Time, len(body.DataPoints))
values := make([]float64, len(body.DataPoints))
for i, p := range body.DataPoints {
times[i] = p.Time
values[i] = p.Value
}
// Create frame and add it to the response
dataResp := backend.DataResponse{
Frames: []*data.Frame{
data.NewFrame(
"response",
data.NewField("time", nil, times),
data.NewField("values", nil, values),
),
},
}
return dataResp, err
}
Other resource requests
Finally, there is an optional type of request that plugins may implement. This is what we call resources. Resources are additional endpoints that the plugin exposes and uses to populate the query editor or config editor. For example, you can use a resource to populate a dropdown menu with a list of available tables in a database.
In a frontend data source, the plugin should define the resources in the plugin.json file as routes and use the fetch method to get the data. For example:
{
"routes": [
{
"path": "tables",
"url": "https://api.example.com/api/v1/tables",
"method": "GET"
}
]
}
export class DataSource extends DataSourceApi<MyQuery, MyDataSourceOptions> {
async getTables() {
const response = await lastValueFrom(
getBackendSrv().fetch<MetricsResponse>({
url: `${this.url}/tables`,
method: 'GET',
})
);
return response.data;
}
}
To keep things simple, authentication is omitted in this example.
For a backend data source, the plugin should implement the CallResourceHandler interface. This interface should handle the different possible resources. For example:
func NewDatasource(_ context.Context, _ backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return &Datasource{
CallResourceHandler: newResourceHandler(),
}, nil
}
func newResourceHandler() backend.CallResourceHandler {
mux := http.NewServeMux()
mux.HandleFunc("/tables", handleTables)
return httpadapter.New(mux)
}
func handleTables(w http.ResponseWriter, r *http.Request) {
// Get tables
res, err := http.DefaultClient.Get("https://api.example.com/api/v1/tables")
// Handle errors (omited)
body, err := io.ReadAll(res.Body)
// Handle errors (omited)
w.Write(body)
w.WriteHeader(http.StatusOK)
}
To request the resource in the frontend, you can use the methods exposed in the base class DataSourceWithBackend (for example, getResource or postResource):
export class DataSource extends DataSourceWithBackend<MyQuery, MyDataSourceOptions> {
async getTables() {
const response = await this.getResource('tables');
return response;
}
}
Conclusion
This guide covered the main steps to convert a frontend data source into a backend data source. There are a wide variety of plugins so if you have any questions or need help with a specific case, we encourage you to reach out in our Community forum. Contributions to this guide are also welcome.