Help build the future of open source observability software Open positions

Check out the open source projects we support Downloads

Grot cannot remember your choice unless you click the consent notice at the bottom.

New in Grafana 7.0: Data transformations for all visualizations that support queries

New in Grafana 7.0: Data transformations for all visualizations that support queries

11 Jun, 2020 4 min

Among the many new features introduced in Grafana 7.0, one of the most exciting ones is transformations. The concept of data transformation is not new to Grafana, but prior to Grafana 7, it was fairly rigid and available only in the Table panel.

What are transformations?

Transformations are a way to modify the data before it is passed to the visualization. They allow you to filter data or queries, and organize or combine data. On top of that, you can perform reduce calculations or simple math on time series based on the results from the data source of your choice.

Take a look at the table below showing time, temperature, and pressure over time:

Table showing time, temperature, and pressure over time
Table showing time, temperature, and pressure over time

With, for example, the organize fields transformation, you can easily hide, reorder, or rename the result set’s fields. In the table below, the temperature field was renamed to Temp, pressure to Pressure, and the humidity field was hidden.

Renamed table
Renamed table

You can chain multiple transformations together; they are run in a sequence. This means that the result from the previous transformation will be passed to the next one in the transformations list. For example, you can first filter the results set and then apply calculation on the filtered data. The image below shows an example of these transformations. The result set was first filtered (humidity field is filtered out), and then the reduce transformation was applied to calculate the mean and max of the temperature and pressure.

Table showing mean and max of the temperature and pressure
Table showing mean and max of the temperature and pressure

Transformations are available for all visualizations that support queries.

Use cases for transformations

Here are some common use cases in which you may find transformations useful.

Shared query

Grafana comes with a concept of dashboard data source, which allows you to reuse queries from another panel within the same dashboard. For expensive queries, transformations may be very helpful to reduce the load on the data source and perform some of the data processing on the client side. You can have multiple panels in the dashboard that visualize the same data without issuing the queries multiple times.

Legacy data sources

Transformations may be helpful in case of using legacy data sources or data sources that do not have support for aggregations.

Performing arithmetic operations on time series

For data sources not supporting math operations across time series, the Add field from calculation transformation may come handy. It allows you to create a new time series that is a result of simple, binary math operations (sum, subtraction, multiplication, division) between two time series or reduction.

Adding transformations to your data

Transformations are available from the Transform tab in the bottom pane of the panel editor, next to the Queries tab.

  1. Navigate to the panel that you want to add transformations, click the panel title, and then click Edit.

  2. Click the Transform tab.

  3. Click a transformation to select it.

A transformation row appears, allowing you to configure the transformation options. Click Add transformation to apply another transformation. Keep in mind that the next transformation acts on the result set returned by the previous transformation.

To learn more about transformations, please visit the documentation.

How we enabled transformations in Grafana 7.0

Between Grafana 6 and 7, we put a lot of effort into standardizing and unifying the data processing in Grafana. We have introduced a completely new data processing pipeline with data frames at its core. The change of the model under the hood enabled the new architecture that the transformations are part of.

New data processing pipeline
New data processing pipeline

The data that comes from the data source, the result set, is being passed to the transformations engine. After transformations are performed, data options and overrides like units, thresholds, and data links are applied. Lastly, the data is passed to the visualization itself and displayed on the dashboard.

We hope that you are going to love the transformations! We are looking forward to your feedback and creative use cases of this amazing feature.

For more on Grafana 7.0

Read more about the new features in Grafana 7.0 in this blog post, and watch Torkel Ödegaard’s demo from GrafanaCONline here: