Define a folder and alert rule group

Alert rules in Grafana live inside folders. You’ll create a folder first, then define a rule group inside it. A rule group is a named collection of related alert rules that share the same evaluation interval.

To define a folder and rule group, complete the following steps:

  1. Create a file named alerts.tf with a folder and rule group:

    hcl
    resource "grafana_folder" "alerts" {
      title = "Terraform Alerts"
    }
    
    resource "grafana_rule_group" "cpu_alerts" {
      name             = "CPU alerts"
      folder_uid       = grafana_folder.alerts.uid
      interval_seconds = 60
    
      rule {
        name      = "High CPU usage"
        condition = "C"
        for       = "5m"
    
        data {
          ref_id         = "A"
          datasource_uid = "<YOUR_DATASOURCE_UID>"
    
          relative_time_range {
            from = 600
            to   = 0
          }
    
          model = jsonencode({
            expr = "100 - (avg(rate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)"
          })
        }
    
        data {
          ref_id         = "B"
          datasource_uid = "__expr__"
    
          relative_time_range {
            from = 0
            to   = 0
          }
    
          model = jsonencode({
            type       = "reduce"
            expression = "A"
            reducer    = "last"
          })
        }
    
        data {
          ref_id         = "C"
          datasource_uid = "__expr__"
    
          relative_time_range {
            from = 0
            to   = 0
          }
    
          model = jsonencode({
            type       = "threshold"
            expression = "B"
            conditions = [{
              evaluator = {
                type   = "gt"
                params = [80]
              }
            }]
          })
        }
    
        labels = {
          severity = "warning"
        }
    
        annotations = {
          summary     = "CPU usage is above 80%"
          description = "CPU usage has been above 80% for 5 minutes."
        }
      }
    }
  2. Replace <YOUR_DATASOURCE_UID> with the UID of a Prometheus data source in your Grafana instance. You can find the UID in Grafana under Connections > Data sources, then click your data source and note the UID from the URL or the settings page.

Note: If you don’t have a Prometheus data source, you can use grafana-testdata-datasource as the datasource_uid and adjust the query model to use TestData expressions instead.

How the data blocks work

The three data blocks form a query pipeline:

Blockref_idWhat it does
QueryAFetches CPU idle percentage from Prometheus
ReduceBReduces the time series to a single value (last)
ThresholdCFires the alert when the value exceeds 80

The condition = "C" on the rule tells Grafana which data block determines when the alert fires.

In the next milestone, you add a contact point to receive alert notifications.


page 5 of 10