New in Grafana 7.0: Data transformations for all visualizations that support queries
Among the many new features introduced in Grafana 7.0, one of the most exciting ones is transformations. The concept of data transformation is not new to Grafana, but prior to Grafana 7, it was fairly rigid and available only in the Table panel.
What are transformations?
Transformations are a way to modify the data before it is passed to the visualization. They allow you to filter data or queries, and organize or combine data. On top of that, you can perform reduce calculations or simple math on time series based on the results from the data source of your choice.
Take a look at the table below showing time, temperature, and pressure over time:
With, for example, the organize fields transformation, you can easily hide, reorder, or rename the result set’s fields. In the table below, the temperature field was renamed to Temp, pressure to Pressure, and the humidity field was hidden.
You can chain multiple transformations together; they are run in a sequence. This means that the result from the previous transformation will be passed to the next one in the transformations list. For example, you can first filter the results set and then apply calculation on the filtered data. The image below shows an example of these transformations. The result set was first filtered (humidity field is filtered out), and then the reduce transformation was applied to calculate the mean and max of the temperature and pressure.
Transformations are available for all visualizations that support queries.
Use cases for transformations
Here are some common use cases in which you may find transformations useful.
Shared query
Grafana comes with a concept of dashboard data source, which allows you to reuse queries from another panel within the same dashboard. For expensive queries, transformations may be very helpful to reduce the load on the data source and perform some of the data processing on the client side. You can have multiple panels in the dashboard that visualize the same data without issuing the queries multiple times.
Legacy data sources
Transformations may be helpful in case of using legacy data sources or data sources that do not have support for aggregations.
Performing arithmetic operations on time series
For data sources not supporting math operations across time series, the Add field from calculation transformation may come handy. It allows you to create a new time series that is a result of simple, binary math operations (sum, subtraction, multiplication, division) between two time series or reduction.
Adding transformations to your data
Transformations are available from the Transform tab in the bottom pane of the panel editor, next to the Queries tab.
Navigate to the panel that you want to add transformations, click the panel title, and then click Edit.
Click the Transform tab.
Click a transformation to select it.
A transformation row appears, allowing you to configure the transformation options. Click Add transformation to apply another transformation. Keep in mind that the next transformation acts on the result set returned by the previous transformation.
To learn more about transformations, please visit the documentation.
How we enabled transformations in Grafana 7.0
Between Grafana 6 and 7, we put a lot of effort into standardizing and unifying the data processing in Grafana. We have introduced a completely new data processing pipeline with data frames at its core. The change of the model under the hood enabled the new architecture that the transformations are part of.
The data that comes from the data source, the result set, is being passed to the transformations engine. After transformations are performed, data options and overrides like units, thresholds, and data links are applied. Lastly, the data is passed to the visualization itself and displayed on the dashboard.
We hope that you are going to love the transformations! We are looking forward to your feedback and creative use cases of this amazing feature.
For more on Grafana 7.0
Read more about the new features in Grafana 7.0 in this blog post, and watch Torkel Ödegaard’s demo from GrafanaCONline here:
There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, check out our blog!