New in Grafana 7.0: Data transformations for all visualizations that support queries

Published: 11 Jun 2020 RSS

Among the many new features introduced in Grafana 7.0, one of the most exciting ones is transformations. The concept of data transformation is not new to Grafana, but prior to Grafana 7, it was fairly rigid and available only in the Table panel.

What are transformations?

Transformations are a way to modify the data before it is passed to the visualization. They allow you to filter data or queries, and organize or combine data. On top of that, you can perform reduce calculations or simple math on time series based on the results from the data source of your choice.

Take a look at the table below showing time, temperature, and pressure over time:

Table showing time, temperature, and pressure over time

With, for example, the organize fields transformation, you can easily hide, reorder, or rename the result set’s fields. In the table below, the temperature field was renamed to Temp, pressure to Pressure, and the humidity field was hidden.

Renamed table

You can chain multiple transformations together; they are run in a sequence. This means that the result from the previous transformation will be passed to the next one in the transformations list. For example, you can first filter the results set and then apply calculation on the filtered data. The image below shows an example of these transformations. The result set was first filtered (humidity field is filtered out), and then the reduce transformation was applied to calculate the mean and max of the temperature and pressure.

Table showing mean and max of the temperature and pressure

Transformations are available for all visualizations that support queries.

Use cases for transformations

Here are some common use cases in which you may find transformations useful.

Shared query

Grafana comes with a concept of dashboard data source, which allows you to reuse queries from another panel within the same dashboard. For expensive queries, transformations may be very helpful to reduce the load on the data source and perform some of the data processing on the client side. You can have multiple panels in the dashboard that visualize the same data without issuing the queries multiple times.

Legacy data sources

Transformations may be helpful in case of using legacy data sources or data sources that do not have support for aggregations.

Performing arithmetic operations on time series

For data sources not supporting math operations across time series, the Add field from calculation transformation may come handy. It allows you to create a new time series that is a result of simple, binary math operations (sum, subtraction, multiplication, division) between two time series or reduction.

Adding transformations to your data

Transformations are available from the Transform tab in the bottom pane of the panel editor, next to the Queries tab.

  1. Navigate to the panel that you want to add transformations, click the panel title, and then click Edit.

  2. Click the Transform tab.

  3. Click a transformation to select it.

A transformation row appears, allowing you to configure the transformation options. Click Add transformation to apply another transformation. Keep in mind that the next transformation acts on the result set returned by the previous transformation.

To learn more about transformations, please visit the documentation.

How we enabled transformations in Grafana 7.0

Between Grafana 6 and 7, we put a lot of effort into standardizing and unifying the data processing in Grafana. We have introduced a completely new data processing pipeline with data frames at its core. The change of the model under the hood enabled the new architecture that the transformations are part of.

New data processing pipeline

The data that comes from the data source, the result set, is being passed to the transformations engine. After transformations are performed, data options and overrides like units, thresholds, and data links are applied. Lastly, the data is passed to the visualization itself and displayed on the dashboard.

We hope that you are going to love the transformations! We are looking forward to your feedback and creative use cases of this amazing feature.

For more on Grafana 7.0

Read more about the new features in Grafana 7.0 in this blog post, and watch Torkel Ödegaard’s demo from GrafanaCONline here:

Related Posts

In this tutorial, Developer Advocate Marcus Olsson reviews the Grafana feature that allows you to dynamically add and remove entire panels.
How Grafana Labs leverages the regexp syntax package to simplify and improve Loki regex performance
When the caller ID and spam-blocking company Hiya decided they needed a monitoring service that would allow them to filter and aggregate metrics and save money, they said hello to Grafana Cloud.

Related Case Studies

After trying to DIY, Wix embraces Grafana Cloud

Metrics is an important part of Wix’s culture, so Grafana Cloud was chosen to monitor mission-critical systems.

"It doesn’t make sense, price-to-performance, to do it ourselves, so we were looking for a fully-managed solution from a team that had experience running monitoring at scale."
– Alex Ulstein, Head of Monitoring, Wix

DigitalOcean gains new insight with Grafana visualizations

The company relies on Grafana to be the consolidated data visualization and dashboard solution for sharing data.

"Grafana produces beautiful graphs we can send to our customers, works with our Chef deployment process, and is all hosted in-house."
– David Byrd, Product Manager, DigitalOcean

Grafana enhances end user experience for Apica Systems

The company uses Grafana alongside its SaaS product to detect availability and performance issues before they affect users.

"There have been all kinds of dashboard solutions, but Grafana changed that game in terms of being extremely dynamic and hooking in whatever you like."
– Sven Hammar, CEO, Apica Systems