Grafana Simple gRPC Datasource Plugin

What is this plugin?

This back-end Grafana datasource plugin provides a user-friendly grafana experience with only a handful simple and generic parameters to configure. It comes with a dedicated API specification that requires implementation in the data provider's back-end. Implementing this API helps to decouple the front-end visualisation solution from the back-end data-layer implementation, leaving developers with the necessary freedom to update and improve the back-end without breaking the end-user experience.

The protobuf API specification can be found in the pkg/proto directory. On configuring the datasource plugin, the end-user provides an endpoint URL and optionally an API key too. The datasource will attempt to establish a gRPC connection and emit calls to the given endpoint according to the API specification.

For more information on gRPC or protobuf, see the gRPC docs.

Why gRPC?

gRPC is a fast & efficient framework for inter-service communication and provides a fool-proof and streamlined workflow for API implementation through protobuf.

gRPC also supports all essential streaming capabilities, which can be implemented in future releases.

Security

The datasource plugin establishes a secure gRPC connection through TLS. Additionally, the datasource supports API-key authorization. The API-key will be included in each API call as part of the call metadata.

Usage

Metric

The variable that is updated with new values as the stream of timeseries datapoints is appended.

Dimension

A dimension is an optional, identifying property of the measure. Each dimension is modeled as a key-value pair. A measure can have zero or many dimensions that collectively uniquely identify it.

Query Type

Getting started

1. start a sample grpc server locally:

docker run -p 50051:50051 innius/sample-grpc-server

1. install the innius-simple-grpc-datasource

2. enable the datasource

   • configure the endpoint localhost:50051

3. configure dashboards

Implement your own backend API

This datasource plugin expects a backend to implement the Simple or the Advanced interface.

The Simple API (GrafanaQueryAPI)

This API provides the following operations:

A sample implementation can be found here.

This API has some limitations:

• it only supports one metric per query
• it does not support variables with multiple options
• it does not support enhanced metadata for metrics (like unit, etc.)
• it does not support flexible query options

The Advanced API (GrafanaQueryAPIV3)

This API provides almost the same operations as the Simple API but with one major difference: it supports multiple metrics for the same query. As a result this API integrates seamlessly with grafana templating capabilities. In addition, it supports enhanced metric metadata, like unit of measure. Another difference is that it supports grafana labels.

The advanced API supports dynamic query options which are defined by the backend system. This makes it possible to tailor the behavior of grafana queries for specific backends. An example of a custom option is the Aggregate of the GetMetricAggregate query. The v1 version of the API has a fixed number of Aggregates, defined by the plugin. It is not possible for a backend system to add a different option. With the V3 API, however, this is supported. Currently an option can be either an Enumeration or a Boolean type.

This API provides the following operations:

A sample implementation can be found here.

Example Use Cases:

• different time series for the same metric with different labels. For example: the temperature measure is a room. The room has four zones: north, south, east and west. The V1 API does not support this unless there are four different metrics defined for each temperature / zone combination. The Advanced API does support this scenario by returning multiple time series for the same metric temperature, each annotated with different label zone.
• different time series for different metrics. For example: a room has multiple temperature sensors. The V1 API supports this by defining multiple queries for each metric. The Advanced API can do this with a single query.

Important Note: in order to use the Advanced API the backend server needs to support gRPC Reflection. The plugin uses this to determine if a backend supports the V2 or V3 protocol. If not supported it falls back on the Simple API implementation.

Please note gRPC is programming language agnostic which makes it possible to implement a backend in the language of your choice. Checkout the gRPC documentation of your language.

Changes between (GrafanaQueryAPIV2) and (GravanaQueryAPIV3)

The most important difference is that the Aggregate types of the V2 API are not available by the V3 API unless they are defined in the backend.

The backend code has to implement something like this:

const (
    // this id is important because it matches the current v2 aggregate type option 
    AggregationTypeOptionID = iota
    // these enum values are important because they match the values of the V2 options 
    AggregationTypeAverage = 0
    AggregationTypeMax     = 1
    AggregationTypeMin     = 2
    AggregationTypeCount   = 3
)
func (backend *BackendServerV3) GetQueryOptions(ctx context.Context, in *v3.GetOptionsRequest) (*v3.GetOptionsResponse, error) {
var Options []*v3.Option
switch in.GetQueryType() {
case v3.GetOptionsRequest_GetMetricAggregate:
Options = append(Options, []*v3.Option{
{
Id:          strconv.Itoa(AggregationTypeOptionID),
Label:       “Aggregate”,
Description: “Aggregate the query results”,
Type:        v3.Option_Enum,
EnumValues: []*v3.EnumValue{
{Label: “Average”, Description: “Calculate the average of the values”, Id: strconv.Itoa(AggregationTypeAverage)},
{Label: “Min”, Description: “Calculate the minimum of the values”, Id: strconv.Itoa(AggregationTypeMin)},
{Label: “Max”, Description: “Calculate the maximum of the values”, Id: strconv.Itoa(AggregationTypeMax)},
{Label: “Count”, Description: “Calculate the sum of the values”, Id: strconv.Itoa(AggregationTypeCount)},
},
},
}…)
case v3.GetOptionsRequest_GetMetricValue:
return &v3.GetOptionsResponse{}, nil
case v3.GetOptionsRequest_GetMetricHistory:
return &v3.GetOptionsResponse{}, nil
}
return &v3.GetOptionsResponse{Options: Options}, nil
}

A sample implementation of the V3 backend can be found here

Features

• select multiple metrics in one query
• flexible dimension selection
• integrated with Grafana variables and templating
• allow backend systems to provided additional metadata, like value mappings, unit of measure, etc.
• supports notifications
• supports pagination
• supports retries for grpc calls if backend server is at maximum capacity
• allow backend systems to define custom query options.

Roadmap

• support annotations
• support streaming queries