Plugins 〉Flow

Panel
community

Flow

  • Overview
  • Installation
  • Change log
  • Related content

Flow

SVG flowchart visualization

Dynamic JSON Badge Downloads License

The Flow plugin provides side-by-side metric visualization on SVG diagrams. You provide the SVG diagram along with configuration on how you want the diagram to behave. The panel then combines these with your time-series data to bring your diagram to life.

Example 1

example1

Example 2

example2

Example 3

example3

Target Audience

Dashboards detailing flow through distributed systems quickly get big and often benefit from some kind of lead in architecture diagram. This panel is aimed at supporting these diagrams. You create the SVG in a rich editor such as draw.io and then associate the data-series and thresholds via yaml config. As well as annotating the diagram with data, you can also add links allowing you to click on the different widgets and dive into an 'exploded' view of the selected component.

Think glass-cockpit with layers of detail; each layer giving ever increasing detail on system health. When creating dashboard suites such as these you often end up with many similar widgets that benefit from sharing the same threshold config. This panel actively supports this by allowing you to share thresholds and links, both within a panel and also across panels. It even goes one step further by allowing you to tailor threshold sets based off of grafana variable selections. This really helps make the thresholds relevant in all situations.

The panel isn't limited to just last-values. We've already paid the price of pulling in a time-range so this panel allows you to see the diagram at any of those points in time. Traverse the timeSlider back and forth to see your flows evolve. There is no grafana overhead for this because the data-series have already been fetched; we're just changing the point-of-interest.

Feature Set

The plugin content is defined with three data files:

  • SVG: This defines the flowchart with controllable widgets identified via the cell IDs.
  • PanelConfig: Defined in yaml, this associates the above cell IDs with time-series, thresholds, links and visualization config.
  • SiteConfig: Defined in yaml, this provides cross-panel shared configuration such as thresholds, links, colors, etc.

The data files can be pasted into the panel, but they can alternatively be loaded from external URLs such as github. When first rendered the panel shows the flowchart at last-values. i.e. the end of the time-window. But this is movable. A Time-Slider at the bottom of the panel allows the full timeseries to be traversed so you can see how the flow evolves over the time window.

The supported visualizations are:

  • label content
  • label color
  • fill color
  • links

Note that tooltips have no inbuilt support in the panel, but can be defined in the SVG and will appear in the panel on-hover courtesy of the underlying framework. To set a tooltip in draw.io, select widget -> Edit -> Edit Tooltip; or select widget -> <ALT><SHFT>T.

Getting Started

Create a Flow Panel in your dashboard. The initial setup loads the first example from this repo via urls. It also enables test-timeseries-enrichment. These test timeseries ensure the initial dashboard has some data to show. It gives you a working example straight out of the box.

Explore

  • Scale:
    • Change the size and shape of the grafana panel. See the SVG automatically scale to fill the content whilst maintaining aspect ratio.
  • Time Slider:
    • Drag the time-slider back and forth. See the flow labels, label colors, fill colors change according to the defined thresholds.
    • Open the panel editor and disable the time slider to see the SVG expand into the additional space and revert to last values.
  • Links:
    • Move the mouse over a cell. See the cursor change to a pointer. Click and dive off to the associated link.
    • Move the mouse over the 'Active Workers' cell. note the mouse doesn't change as there is no associated link.
  • SVG and Config References:
  • Test Data Generation:
    • Deselect generation and see the SVG change to unannotated. Reselect and see the color and values come back.
  • Debugging: Note these steps all need you to open your browser console.
    • Data: Press the button and expand the two log lines of Debugging Data raw and Debugging Data enriched. The raw view is the timeseries grafana has sent to the panel. The enriched view is the data after test-data has been included. In the enriched view every used time-series has a corresponding time series and the time series also shows the valuesIndex that has been chosen based on the time-slider position.
    • Mappings: Press the button and you'll get log lines for abstract and verbose. The abstract line shows how SVG cell IDs have been assigned to time-series. The verbose line gives all of the SVG mapping information.
    • Units: Press the button and expand out the log line showing the Category -> Subcategory -> {text, value} for every potential unit. The unit strings you enter in the yaml config are the values shown here.
    • Colors: Press the button and see the available grafana theme hues. Each hue has mutiple shades. These color names can be used directly in the yaml. Alternatively you can enter short hex, long hex, rgb or hsl. All of these formats will be recognised and so will also work with gradient selected. If you want to enter html color names instead, they will be passed through to the SVG but they won't be understood by the panel and so gradient will have no effect.
    • SVG: Press the button and see the serialized svg that is being displayed. Copy it out to compare to the passed in svg template or save away as a snapshot image.
    • Timings: Press the button to instrument the next render pass. This allows you to see where the time is going:
      • toDataFrame: The panel only understands the dataframe format so the first step is to convert incoming data into this format. If you aren't sure what format you are receiving, run the query in Query inspector and look at the response. i.e. a dataframe will contain a fields array whilst a time-series will contain a datapoints or points array. Note that moving the timeSlider requires a fresh render pass and the panel has no way of telling if the data received is the result of a fresh fetch. As such this step has to be repeated each time the timeSlider moves. However, it's only expensive if the incoming data is not in a dataframe format.
      • transform: This converts the dataframes to the format needed by this panels interpolation. It's a light weight array restructuring.
      • seriesExtend: This adds in the test time-series. You will only see this if test time series are enabled in the panel options.
      • seriesInterpolate: This interpolates each of the time arrays to determine the correct index for the given timeSlider position.
      • svgUpdate: This applies the label and threshold settings to the document.
      • createJsx: This creates the JSX from the svg document html.

Using draw.io to create your SVG

Any tool can be used to maintain your SVG but draw.io is the tool used for all of these examples. draw.io is available online and also as an app download. When launching the online version remember to include the ?p=svgdata as an argument. This loads the svgdata plugin and ensures your cell IDs will be saved down in your SVG file. Via the app, you get the same by going to Extras->plugins->Add->svgdata.js. With the plugin, any ID can be edited in-situ by selecting the cell then Edit->Edit Data->Double click on the ID value->edit the ID.

Note that draw.io IDs under the hood get given a preamble of cell-. To make this easier the panelConfig yaml allows the preamble to be set so the actual yaml data can reference the IDs as you've entered them. You can see the difference in the Mappings data as it shows your ID as well as the full svg ID.

The SVG can be entered directly into the panel or it can be referenced via url.

Using YAML to configure your Flowcharts

YAML offers a few useful features over other config formats. (1) It can be commented. (2) It supports anchors and aliases. Both of these features are really useful for long term maintainability. The yaml files can be entered directly into the panel or they can be referenced via url.

The panelConfig details the set of SVG IDs that are going to be driven and how they are going to be driven. It's particular to the panel.

The siteConfig details common settings that you are using across a suite of panels. It provides a way to define a system level theme, so all the panels respond in a similar way and can be maintained with single touch points. You don't have to use a siteConfig. All parameters can be defined in the panelConfig. The only reason for using a siteConfig is for when you are maintaining a suite of panels. The panelConfig and siteConfig urls will have grafana variables replaced prior to fetching the files. This gives you some level of multi-site configurability.

Grafana Variables can also be used to scale threshold definitions. i.e. if you have an environment variable you could scale the thresholds differently in dev and prod.

These links take you to yaml files where each of the settings are documented:

First Panel

Once your familiar with the controls creating your first proper panel boils down to this process:

  • In draw.io, create an SVG and edit the cell IDs to meaningful names. Keep it minimal in the first instance.
  • Copy an example panelConfig as that's setup with cellIds that are using testData. Edit the cellIDs to match your SVG IDs.
  • Create your Flow panel and set it to be driven from your SVG and your panelConfig.
  • See your SVG responding in the correct units to the test data.
  • Develop your time-series query in its own time-series panel until you are happy with the result. Remember to alias the time-series to a meaningful name as you will use in your Flow panel.
  • Copy your query into the Flow panel.
  • Press 'Debugging Data' to see your query time-series coming through to the panel.
  • Change your panel config dataRef to map the cell ID to your time-series name.
  • Use the time-slider to see your value being correctly echoed in the SVG.
  • Rinse / Repeat.

Want to make changes?

Go to the Grafana Getting Started guide to get going with downloading a fork and setting up your test environment. Once up you'll find dashboards available (sourced from the provisioning directory) that act as demonstrators for the functionality.

Installing Flow on Grafana Cloud:

For more information, visit the docs on plugin installation.

Changelog

1.14.0

Threshold Support for Text Data

This adds the ability to configure thresholds using regex patterns for text data. New config terms in siteConfig and panelConfig:

  • siteConfig:thresholdPatterns
  • panelConfig:cells.cell-name.labelColor.thresholdPatterns
  • panelConfig:cells.cell-name.labelColor.thresholdPatternsRef
  • panelConfig:cells.cell-name.strokeColor.thresholdPatterns
  • panelConfig:cells.cell-name.strokeColor.thresholdPatternsRef
  • panelConfig:cells.cell-name.fillColor.thresholdPatterns
  • panelConfig:cells.cell-name.fillColor.thresholdPatternsRef

Link in the Same Tab

This adds the ability to configure links to open in the same tab rather than a new tab. It can be set at the cell, panel or site level. New config terms in siteConfig and panelConfig:

  • siteConfig:linkWindow
  • panelConfig:linkWindow
  • panelConfig:cells.cell-name.link.sameTab

Link Variables

This adds the ability to define link url substitutions to simplify oft-repeated base urls or argument sets. New config terms in siteConfig and panelConfig:

  • linkVariables

Zoom Pan Pinch Wheel Activation

This adds the ability to require an activation key such as 'Alt' to be pressed in order to enter panel zoom. There's a clash of interest between panel zoom and dashboard scroll which can be avoided by using this feature. New config terms in siteConfig and panelConfig:

  • zoomPanPinch

Compound Colors

This feature allows a labelColor, strokeColor or fillColor to be driven based off an array of dataRef inputs. i.e you can choose the worst or the best color based off of a set of inputs. To support this a new field has been added to 'threshold' called 'order'. It defaults to the threshold-array index but can be explicitly driven in yaml. This is the term used to compare thresholds from different dataRefs. On the whole it won't be needed but it's inclusion means no implicit ordering has to cross all threshold definitions. New config terms:

  • cells.cell-name.labelColors
  • cells.cell-name.strokeColors
  • cells.cell-name.fillColors

1.13.0

Fill Level

Adds support for driving fill levels. The yaml defines a data->percentage-fill mapping. Under the hood it's implemented using the SVG clip-path feature. The shapes bounding box is calculated and an additional clip-path rect is added to the SVG. This rect then gets driven based off of the incoming data. New config terms:

  • cells.cell-name.fillLevel

Stroke Color

Adds support for driving stroke color. i.e. coloring the structural lines of shapes and connections. The yaml drive is identical in nature to the fillColor drive. New config terms:

  • cells.cell-name.strokeColor

Grafana Variable Substitution in ValueMappings

Adds support for passing valueMappings text through grafana variable substitution. i.e. to substitute all values of a timeseries with a variable value: cell abc: valueMappings: - {text = "${myVariableName}"}

1.12.0

URL Grafana Variables

Adds support for grafana variables in the SVG URL. Before this was only available in the YAML URLs. It also corrects the variable change detection logic for all url variables to ensure we get a fresh fetch when relevant variables change.

String Data

This adds support for string timeseries for label drives. Color can still be driven by having number timeseries for colors and string timeseries for the label text. This is also compatible with value mappings.

Relative Links

Adds support for relative links. Before only absolute links were available.

1.11.0

Flow Animations

Adds support for draw.io line flow animations. In draw.io lines with 'Flow Animation' selected and the resulting SVG 'exported-as-svg' (not just saved as SVG) results in animated lines that are controllable via 'direction' and 'duration' attributes. This change adds yaml config to drive these animations off of data. When such config is present a play/pause button is always added to the panel to ensure the panel is usable by all. New config terms:

  • cells.cell-name.flowAnimation

As well as provisioning dashboards, the above functionality is demonstrated in example 4:

  • svg: https://raw.githubusercontent.com/andymchugh/andrewbmchugh-flow-panel/main/examples/darkThemeSvg4.svg
  • panelConfig: https://raw.githubusercontent.com/andymchugh/andrewbmchugh-flow-panel/main/examples/panelConfig4.yaml

New test data added to facilitate testing of the flow-animations:

  • test.testDataBaseOffset
  • test.testDataExtendedZero

One Datapoint Interpolation fix

Fixes a bug with timeSeries interpolation which resulted in no data when the timeSeries only had a single datapoint.

Value Mappings Support

This allows you to configure a custom value based on data-match criteria. i.e. if your data is 0, 1, 2 you can use this configuration to replace the 'label' text with 'auto', open', 'closed'. The criteria you can use to match are:

  • exact value match
  • range match
  • partial range match.

New panelConfig terms:

  • cells.cell-name.label.valueMappings
  • cells.cell-name.label.valueMappingsRef

1.10.0

Increases the range of SVG shape support by relaxing the expected DOM element hierachy.

1.9.0

Adds Accessibility in the form of Pan / Zoom to ensure the SVG is readable for the greatest range of people.

1.8.0

Corrects the source of the time-field from the incoming dataFrame. Before it relied on the name being toLower('time'). Now it correctly looks up the type and breaks out on first match with type time. This fixes use with datasources like ElasticSearch where the field name defaults to '@timestamp'.

1.7.0

Improves scaling in all ways but especially the sub 100% scaling so that it now correctly fills the panel maintaining aspect ratio.

Changes the background color definition so that it applies to the available svg panel area rather that the actual svg background.

Adds a new panelConfig field 'unitsPostfix'. This is an optional string that will be postfixed to the label value. It can be used for specifying custom units. In yaml, unicode characters are specified with a \u. i.e. "\u03bc" = Mu. New panel config terms:

  • cells.cell-name.label.unitsPostfix

Adds a new panelConfig field 'condensed'. This mode which uses less vertical space for the controls when both the timeSlider and highlighter are enabled. New panelConfig terms:

  • tagConfig.condensed

1.6.0

Adds resource links for website, license and yaml defs to the plugin landing page.

Fixes x-scaling ratio when the SVG x-dimension is smaller than the available window. Now it scales with the window whereas before it scaled at twice the rate resulting in it being much smaller than necessary.

Adds the ability to drive the SVG background. New panelConfig terms:

  • background.darkThemeColor
  • background.lightThemeColor Can be used with the normal color-names, rbg, hex values. When the relevant term is undefined the background color is not driven.

Extends the datapoint choice from 'last' to 'last' or 'lastNotNull'. 'last' remains the default. When coupled with graphite functions 'keepLastValue' and 'transformNull', 'last' offers the best control over the display. Where those graphite functions are not available 'lastNotNull' offers a consistent display when dealing with sparse data. 'lastNotNull' starts with the same datapoint as 'last' and then walks back in time till a non-null value is found. It's configurable in the panel yaml at the panel level and overridable at the cell level and cell-attribute level. To make this testable a 'test' table has been added to the yaml to allow the testData to be generated in sparse mode. New panel config terms of:

  • datapoint
  • cells.cell-name.datapoint
  • cells.cell-name.label.datapoint
  • cells.cell-name.labelColor.datapoint
  • cells.cell-name.fillColor.datapoint
  • test.testDataSparse

Adds 'Thread highlighting'. Each driven SVG cell can now be given an associated tag set and these tags are selectable from an optional 'Highlighter' legend. This provides a way to bring related terms to the fore to help spot correlations in flow. New panel config terms of:

  • tagConfig
  • cells.tags

1.5.0

Fixes the grafana variable threshold matching to break out on first rule match for a given variable/cell tuple. Before it was continuing through the rule set which meant various negative rules were also needed to achieve the right effect.

Fix rendering of SVGs that contain multi-line text blocks. Before the domPurify step was breaking the content making the image unrenderable. This is now fixed by using PARSER_MEDIA_TYPE: 'application/xhtml+xml' in the domPurify sanitization.

Fix link time forwarding when sourced from the args. Before this worked correctly if from/to arguments were present in the url, but when they were missing it would add them in but set to 'null' resulting in a downstream dashboard have a non-existent time window. Now if the values aren't in the url, they aren't appended to the link.

Expose decimalPoints=auto. Before the default was '0' and there was no way to specify auto. Now setting decimalPoints to null will forward through as 'auto'. There's also now a panel level override 'cellLabelDecimalPoints' that will be applied if the value is not specified at the cell level. To keep back-compatibility, if not specified on the cell or in this panel fallback it will still default to '0'.

1.4.0

Changes the data-series name to be sourced from getFieldDisplayName(field, frame). This is a much more involved grafana backed name resolution that should work with all the various ways grafana allows series to be named. Added to resolve issue raised with a prometheus data query.

1.3.6

Grafana 10.0.0 base.

  • rebases the required dependency to Grafana 10.0.0
  • corrects readme example picture links to work from grafana plugins page
  • starts maintaining this changelog

1.3.4

Grafana 10.0.3 base.

Initial release.