Grafana Cloud Alerts and IRM Machine Learning
Grafana Cloud

Machine Learning

Grafana Machine Learning gives Grafana Cloud users the ability to create predictions of the current or future state of their systems.

To create predictions, you define a source query (the time series to be modeled) and the configuration for the machine learning model. The system will train the model in the background.

Once a model has been successfully trained, you can issue queries to predict the value of the series at different times into the future. The model will also return the confidence bounds for the predicted values.

Over time the model will keep learning new patterns, so it automatically evolves along with your data.

Get started

  • See our Getting started tutorial to see metric predictions in action.
  • Querying will help you craft queries using the machine learning Prometheus data source.
  • Model Configuration documents how you can tune models to improve predictions.
  • Outlier Detection explains how to effectively detect outliers and create Outlier-based alerts.
  • Holidays explains how to account for custom holidays to improve predictions.
  • Sift inspects your infrastructure and surfaces interesting details to help you diagnose issues and improve the overall health of your products.


How do alerts work with Grafana Machine Learning?

Grafana Machine Learning leverages the unified alerting capabilities built into Grafana 8.0. See Grafana Alerting for more information.

Which data sources does Grafana Machine Learning support?

Grafana Machine Learning currently supports Prometheus, Graphite, Loki (metric queries only), Postgres, InfluxDB*, BigQuery, Snowflake, Splunk, MongoDB, Elasticsearch and Datadog data sources.

  • Note: InfluxDB Flux queries need to use macros to replace the time range and interval from Grafana’s context.

What are the usage limits for Machine Learning?

There are a variety of limits to the amount of data you can train or predict on.

  1. 50,000 datapoints for training a series
  2. 100 series per forecast
  3. 10 forecasts per instance

If you need to increase any of these limits please contact us or ask your account executive, support engineer, or technical account manager.

My forecasts are not accurate, what should I do?

Start by looking at the model configuration options, and also think about what your data looks like. For example, if you have strong daily seasonality (traffic increasing at a similar time each day) you may want to increase Daily Fourier Order. Similarly, with strong weekly seasonality, for example lower traffic on the weekends, increasing the Weekly Fourier Order may help.

What should I do if my forecast fails?

First, open up the details for your forecast to get a more detailed error message. If the forecast timed out during training try reducing the resolution or training window in the model configuration. Trainings that fail due to system errors, such as receiving a 5XX error from the backing data source, will automatically be retried.

How do I keep my forecasts up to date?

Forecasts are retrained every day to make sure they remain accurate.

How does billing work?

Grafana Machine Learning is available for all Grafana Cloud accounts.

NOTE: If you would like to raise the limits to push the ML even further, there may be an additional charge. Please get in touch.

Can I use Grafana Machine Learning outside of Grafana Cloud?

Grafana Machine Learning is only available on Grafana Cloud, however, you can use Private Data Connect to create a forecast or outlier detector for data you host.