Menu
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Developer resourcesbreadcrumb arrow Infrastructure as codebreadcrumb arrow Terraformbreadcrumb arrow Manage Asserts in Grafana Cloud using Terraform
Grafana Cloud

Manage Asserts in Grafana Cloud using Terraform

Learn how to use Terraform to manage Grafana Cloud Asserts resources. This guide shows you how to create and manage alert configurations, suppressed assertions, custom model rules, and log configurations using Terraform.

Note

Grafana Cloud Asserts supports Terraform-based configuration for alert configurations, suppressed assertions, custom model rules, and log configurations. These resources use the grafana_asserts_ naming convention in Terraform.

Before you begin

Before you begin, you should have the following:

  • A Grafana Cloud account, as shown in Get started
  • Terraform installed on your machine
  • Administrator permissions in your Grafana instance
  • Asserts enabled in your Grafana Cloud stack

Note

All of the following Terraform configuration files should be saved in the same directory.

Configure the Grafana provider

This Terraform configuration sets up the Grafana provider to provide necessary authentication when managing Asserts resources.

You can reuse a similar setup to the one described in Creating and managing a Grafana Cloud stack using Terraform to set up a service account and a token.

  1. Create a Service account and token in Grafana. To create a new one, refer to Service account tokens.

  2. Create a file named main.tf and add the following:

    terraform 
    terraform {
  required_providers {
    grafana = {
      source  = "grafana/grafana"
      version = ">= 2.9.0"
    }
  }
}

provider "grafana" {
  alias = "asserts"

  url      = "<Stack-URL>"
  auth     = "<Service-account-token>"
  stack_id = "<Stack-ID>"
}

  3. Replace the following field values:

    • <Stack-URL> with the URL of your Grafana stack (for example, https://my-stack.grafana.net/)
    • <Service-account-token> with the service account token that you created
    • <Stack-ID> with your Grafana Cloud stack ID

Note

The stack_id parameter is required for Asserts resources to identify the stack where the resources belong.

Create notification alerts configurations

Notification alerts configurations in Asserts allow you to manage how alerts are processed and routed. You can specify match labels to filter alerts, add custom labels, set duration requirements, and control silencing.

Basic notification alerts configuration

Create a file named alert-configs.tf and add the following:

terraform 
# Basic alert configuration with silencing
resource "grafana_asserts_notification_alerts_config" "prometheus_remote_storage_failures" {
  provider = grafana.asserts

  name = "PrometheusRemoteStorageFailures"

  match_labels = {
    alertname   = "PrometheusRemoteStorageFailures"
    alertgroup  = "prometheus.alerts"
    asserts_env = "prod"
  }

  silenced = true
}

# High severity alert with specific job and context matching
resource "grafana_asserts_notification_alerts_config" "error_buildup_notify" {
  provider = grafana.asserts

  name = "ErrorBuildupNotify"

  match_labels = {
    alertname               = "ErrorBuildup"
    job                     = "acai"
    asserts_request_type    = "inbound"
    asserts_request_context = "/auth"
  }

  silenced = false
}

Notification alerts configuration with additional labels and duration

terraform 
# Alert with additional labels and custom duration
resource "grafana_asserts_notification_alerts_config" "payment_test_alert" {
  provider = grafana.asserts

  name = "PaymentTestAlert"

  match_labels = {
    alertname         = "PaymentTestAlert"
    additional_labels = "asserts_severity=~\"critical\""
    alertgroup        = "alex-k8s-integration-test.alerts"
  }

  alert_labels = {
    testing = "onetwothree"
  }

  duration = "5m"
  silenced = false
}

Latency and performance notification alerts

terraform 
# Latency alert for shipping service
resource "grafana_asserts_notification_alerts_config" "high_shipping_latency" {
  provider = grafana.asserts

  name = "high shipping latency"

  match_labels = {
    alertname            = "LatencyP99ErrorBuildup"
    job                  = "shipping"
    asserts_request_type = "inbound"
  }

  silenced = false
}

# CPU throttling alert with warning severity
resource "grafana_asserts_notification_alerts_config" "cpu_throttling_sustained" {
  provider = grafana.asserts

  name = "CPUThrottlingSustained"

  match_labels = {
    alertname         = "CPUThrottlingSustained"
    additional_labels = "asserts_severity=~\"warning\""
  }

  silenced = true
}

Infrastructure and service notification alerts

terraform 
# Ingress error rate alert
resource "grafana_asserts_notification_alerts_config" "ingress_error" {
  provider = grafana.asserts

  name = "ingress error"

  match_labels = {
    alertname            = "ErrorRatioBreach"
    job                  = "ingress-nginx-controller-metrics"
    asserts_request_type = "inbound"
  }

  silenced = false
}

# MySQL Galera cluster alert
resource "grafana_asserts_notification_alerts_config" "mysql_galera_not_ready" {
  provider = grafana.asserts

  name = "MySQLGaleraNotReady"

  match_labels = {
    alertname = "MySQLGaleraNotReady"
  }

  silenced = false
}

Create suppressed assertions configurations

Suppressed assertions configurations allow you to disable specific alerts or assertions based on label matching. This is useful for maintenance windows, test environments, or when you want to temporarily suppress certain types of alerts.

Basic suppressed assertions configuration

Create a file named suppressed-assertions.tf and add the following:

terraform 
# Basic suppressed alert configuration for maintenance
resource "grafana_asserts_suppressed_assertions_config" "maintenance_window" {
  provider = grafana.asserts

  name = "MaintenanceWindow"

  match_labels = {
    service     = "api-service"
    maintenance = "true"
  }
}

# Suppress specific alertname during deployment
resource "grafana_asserts_suppressed_assertions_config" "deployment_suppression" {
  provider = grafana.asserts

  name = "DeploymentSuppression"

  match_labels = {
    alertname = "HighLatency"
    job       = "web-service"
    env       = "staging"
  }
}

# Suppress alerts for specific test environment
resource "grafana_asserts_suppressed_assertions_config" "test_environment_suppression" {
  provider = grafana.asserts

  name = "TestEnvironmentSuppression"

  match_labels = {
    alertgroup  = "test.alerts"
    environment = "test"
  }
}

Service-specific suppression configurations

terraform 
# Suppress alerts for specific services during maintenance
resource "grafana_asserts_suppressed_assertions_config" "api_service_maintenance" {
  provider = grafana.asserts

  name = "APIServiceMaintenance"

  match_labels = {
    service     = "api-gateway"
    job         = "api-gateway"
    maintenance = "scheduled"
  }
}

# Suppress database alerts during backup operations
resource "grafana_asserts_suppressed_assertions_config" "database_backup" {
  provider = grafana.asserts

  name = "DatabaseBackupSuppression"

  match_labels = {
    service     = "postgresql"
    job         = "postgres-exporter"
    backup_mode = "active"
  }
}

# Suppress monitoring system alerts during updates
resource "grafana_asserts_suppressed_assertions_config" "monitoring_update" {
  provider = grafana.asserts

  name = "MonitoringSystemUpdate"

  match_labels = {
    service = "prometheus"
    job     = "prometheus"
    update  = "in_progress"
  }
}

Environment and team-based suppression

terraform 
# Suppress all alerts for development environment
resource "grafana_asserts_suppressed_assertions_config" "dev_environment" {
  provider = grafana.asserts

  name = "DevelopmentEnvironmentSuppression"

  match_labels = {
    environment = "development"
    team        = "platform"
  }
}

# Suppress alerts for specific team during their maintenance window
resource "grafana_asserts_suppressed_assertions_config" "team_maintenance" {
  provider = grafana.asserts

  name = "TeamMaintenanceWindow"

  match_labels = {
    team        = "backend"
    maintenance = "team_scheduled"
    timezone    = "UTC"
  }
}

# Suppress alerts for staging environment during testing
resource "grafana_asserts_suppressed_assertions_config" "staging_testing" {
  provider = grafana.asserts

  name = "StagingTestingSuppression"

  match_labels = {
    environment = "staging"
    testing     = "automated"
    job         = "integration-tests"
  }
}

Alert type and severity-based suppression

terraform 
# Suppress low severity alerts during business hours
resource "grafana_asserts_suppressed_assertions_config" "low_severity_business_hours" {
  provider = grafana.asserts

  name = "LowSeverityBusinessHours"

  match_labels = {
    severity = "warning"
    timezone = "business_hours"
  }
}

# Suppress specific alert types during known issues
resource "grafana_asserts_suppressed_assertions_config" "known_issue_suppression" {
  provider = grafana.asserts

  name = "KnownIssueSuppression"

  match_labels = {
    alertname = "HighMemoryUsage"
    service   = "legacy-service"
    issue_id  = "LEG-123"
  }
}

# Suppress infrastructure alerts during planned maintenance
resource "grafana_asserts_suppressed_assertions_config" "infrastructure_maintenance" {
  provider = grafana.asserts

  name = "InfrastructureMaintenance"

  match_labels = {
    alertgroup = "infrastructure.alerts"
    maintenance_type = "planned"
    affected_services = "all"
  }
}

Complex multi-label suppression

terraform 
# Complex suppression for multi-service deployments
resource "grafana_asserts_suppressed_assertions_config" "multi_service_deployment" {
  provider = grafana.asserts

  name = "MultiServiceDeploymentSuppression"

  match_labels = {
    deployment_id = "deploy-2024-01-15"
    services      = "api,worker,frontend"
    environment   = "production"
    deployment_type = "blue_green"
  }
}

# Suppress alerts for specific cluster during maintenance
resource "grafana_asserts_suppressed_assertions_config" "cluster_maintenance" {
  provider = grafana.asserts

  name = "ClusterMaintenanceSuppression"

  match_labels = {
    cluster     = "production-cluster-1"
    maintenance = "cluster_upgrade"
    affected_nodes = "all"
    estimated_duration = "2h"
  }
}

# Suppress alerts for specific region during network issues
resource "grafana_asserts_suppressed_assertions_config" "regional_network_issue" {
  provider = grafana.asserts

  name = "RegionalNetworkIssueSuppression"

  match_labels = {
    region      = "us-west-2"
    issue_type  = "network"
    affected_services = "external_dependencies"
    incident_id = "NET-456"
  }
}

Create custom model rules

Custom model rules in Asserts allow you to define how entities are discovered and modeled based on Prometheus queries. These rules enable you to create custom entity types, define their relationships, and specify how they should be enriched with additional data.

Basic custom model rules

Create a file named custom-model-rules.tf and add the following:

terraform 
# Basic custom model rule for services
resource "grafana_asserts_custom_model_rules" "basic_service" {
  provider = grafana.asserts

  name = "basic-service-model"

  rules {
    entity {
      type = "Service"
      name = "service"

      defined_by {
        query = "up{job!=''}"
        label_values = {
          service = "job"
        }
        literals = {
          _source = "up_query"
        }
      }
    }
  }
}

Advanced service model with scope and lookup

terraform 
# Advanced service model with environment scoping
resource "grafana_asserts_custom_model_rules" "advanced_service" {
  provider = grafana.asserts

  name = "advanced-service-model"

  rules {
    entity {
      type = "Service"
      name = "workload | service | job"

      scope = {
        namespace = "namespace"
        env       = "asserts_env"
        site      = "asserts_site"
      }

      lookup = {
        workload  = "workload | deployment | statefulset | daemonset | replicaset"
        service   = "service"
        job       = "job"
        proxy_job = "job"
      }

      defined_by {
        query = "up{job!='', asserts_env!=''}"
        label_values = {
          service     = "service"
          job         = "job"
          workload    = "workload"
          namespace   = "namespace"
        }
        literals = {
          _source = "up_with_workload"
        }
      }

      defined_by {
        query    = "up{job='maintenance'}"
        disabled = true
      }
    }
  }
}

Multi-entity model configuration

terraform 
# Multiple entity types in a single model
resource "grafana_asserts_custom_model_rules" "multi_entity" {
  provider = grafana.asserts

  name = "kubernetes-entities"

  rules {
    # Service entity
    entity {
      type = "Service"
      name = "service"

      scope = {
        namespace = "namespace"
        cluster   = "cluster"
      }

      defined_by {
        query = "up{service!=''}"
        label_values = {
          service   = "service"
          namespace = "namespace"
          cluster   = "cluster"
        }
      }
    }

    # Pod entity
    entity {
      type = "Pod"
      name = "pod"

      scope = {
        namespace = "namespace"
        cluster   = "cluster"
      }

      lookup = {
        service   = "service"
        workload  = "workload"
      }

      defined_by {
        query = "kube_pod_info{pod!=''}"
        label_values = {
          pod       = "pod"
          namespace = "namespace"
          cluster   = "cluster"
          service   = "service"
        }
        literals = {
          _entity_type = "pod"
        }
      }
    }

    # Namespace entity
    entity {
      type = "Namespace"
      name = "namespace"

      scope = {
        cluster = "cluster"
      }

      defined_by {
        query = "kube_namespace_status_phase{namespace!=''}"
        label_values = {
          namespace = "namespace"
          cluster   = "cluster"
        }
      }
    }
  }
}

Complex entity with enrichment

terraform 
# Service entity with enrichment from multiple sources
resource "grafana_asserts_custom_model_rules" "enriched_service" {
  provider = grafana.asserts

  name = "enriched-service-model"

  rules {
    entity {
      type = "Service"
      name = "service"

      enriched_by = [
        "prometheus_metrics",
        "kubernetes_metadata",
        "application_logs"
      ]

      scope = {
        environment = "asserts_env"
        region      = "asserts_site"
        team        = "team"
      }

      lookup = {
        deployment = "workload"
        pod        = "pod"
        container  = "container"
      }

      # Primary definition from service up metrics
      defined_by {
        query = "up{service!='', asserts_env!=''}"
        label_values = {
          service     = "service"
          environment = "asserts_env"
          region      = "asserts_site"
          team        = "team"
        }
        literals = {
          _primary_source = "service_up"
        }
      }

      # Secondary definition from application metrics
      defined_by {
        query = "http_requests_total{service!=''}"
        label_values = {
          service     = "service"
          environment = "environment"
          version     = "version"
        }
        literals = {
          _secondary_source = "http_metrics"
        }
      }

      # Disabled definition for testing
      defined_by {
        query    = "test_metric{service!=''}"
        disabled = true
      }
    }
  }
}

Database and infrastructure entities

terraform 
# Database and infrastructure entity models
resource "grafana_asserts_custom_model_rules" "infrastructure" {
  provider = grafana.asserts

  name = "infrastructure-entities"

  rules {
    # Database entity
    entity {
      type = "Database"
      name = "database_instance"

      scope = {
        environment = "env"
        region      = "region"
      }

      lookup = {
        host     = "instance"
        port     = "port"
        db_name  = "database"
      }

      defined_by {
        query = "mysql_up{instance!=''}"
        label_values = {
          database_instance = "instance"
          database         = "database"
          env             = "environment"
          region          = "region"
        }
        literals = {
          _db_type = "mysql"
        }
        metric_value = "1"
      }

      defined_by {
        query = "postgres_up{instance!=''}"
        label_values = {
          database_instance = "instance"
          database         = "datname"
          env             = "environment"
        }
        literals = {
          _db_type = "postgresql"
        }
      }
    }

    # Load balancer entity
    entity {
      type = "LoadBalancer"
      name = "lb_instance"

      scope = {
        environment = "env"
      }

      defined_by {
        query = "haproxy_up{proxy!=''}"
        label_values = {
          lb_instance = "instance"
          proxy      = "proxy"
          env        = "environment"
        }
        literals = {
          _lb_type = "haproxy"
        }
      }
    }
  }
}

Create log configurations

Log configurations in Asserts allow you to define how log data is queried and correlated with entities. You can specify data sources, entity matching rules, label mappings, and filtering options for spans and traces.

Basic log configuration

Create a file named log-configs.tf and add the following:

terraform 
# Basic log configuration for services
resource "grafana_asserts_log_config" "production" {
  provider = grafana.asserts

  name            = "production"
  priority        = 1000
  default_config  = false
  data_source_uid = "grafanacloud-logs"
  error_label     = "error"

  match {
    property = "asserts_entity_type"
    op       = "EQUALS"
    values   = ["Service"]
  }

  match {
    property = "environment"
    op       = "EQUALS"
    values   = ["production", "staging"]
  }

  entity_property_to_log_label_mapping = {
    "otel_namespace" = "service_namespace"
    "otel_service"   = "service_name"
    "environment"    = "env"
    "site"           = "region"
  }

  filter_by_span_id  = true
  filter_by_trace_id = true
}

Log configuration with multiple match rules

terraform 
# Development environment log configuration
resource "grafana_asserts_log_config" "development" {
  provider = grafana.asserts

  name            = "development"
  priority        = 2000
  default_config  = true
  data_source_uid = "elasticsearch-dev"
  error_label     = "error"

  match {
    property = "asserts_entity_type"
    op       = "EQUALS"
    values   = ["Service"]
  }

  match {
    property = "environment"
    op       = "EQUALS"
    values   = ["development", "testing"]
  }

  match {
    property = "site"
    op       = "EQUALS"
    values   = ["us-east-1"]
  }

  match {
    property = "service"
    op       = "EQUALS"
    values   = ["api"]
  }

  entity_property_to_log_label_mapping = {
    "otel_namespace" = "service_namespace"
    "otel_service"   = "service_name"
    "environment"    = "env"
    "site"           = "region"
    "service"        = "app"
  }

  filter_by_span_id  = true
  filter_by_trace_id = true
}

Minimal log configuration

terraform 
# Minimal configuration for all entities
resource "grafana_asserts_log_config" "minimal" {
  provider = grafana.asserts

  name            = "minimal"
  priority        = 3000
  default_config  = false
  data_source_uid = "loki-minimal"

  match {
    property = "asserts_entity_type"
    op       = "IS_NOT_NULL"
    values   = []
  }
}

Advanced log configuration with complex matching

terraform 
# Advanced configuration with multiple operations
resource "grafana_asserts_log_config" "advanced" {
  provider = grafana.asserts

  name            = "advanced"
  priority        = 1500
  default_config  = false
  data_source_uid = "loki-advanced"
  error_label     = "level"

  match {
    property = "service_type"
    op       = "CONTAINS"
    values   = ["web", "api"]
  }

  match {
    property = "environment"
    op       = "NOT_EQUALS"
    values   = ["test"]
  }

  match {
    property = "team"
    op       = "IS_NOT_NULL"
    values   = []
  }

  entity_property_to_log_label_mapping = {
    "service_type"   = "type"
    "team"           = "owner"
    "environment"    = "env"
    "version"        = "app_version"
  }

  filter_by_span_id  = true
  filter_by_trace_id = false
}

Resource reference

The following configurations provide more details and examples for working with the Grafana API.

grafana_asserts_notification_alerts_config

Manage Asserts notification alerts configurations through the Grafana API.

Arguments

NameTypeRequiredDescription
namestringYesThe name of the notification alerts configuration. This field is immutable and forces recreation if changed.
match_labelsmap(string)NoLabels to match for this notification alerts configuration. Used to filter which alerts this configuration applies to.
alert_labelsmap(string)NoLabels to add to alerts generated by this notification alerts configuration.
durationstringNoDuration for which the condition must be true before firing (for example, ‘5m’, ’30s’). Maps to ‘for’ in Asserts API.
silencedboolNoWhether this notification alerts configuration is silenced. Defaults to false.

Example

terraform 
resource "grafana_asserts_notification_alerts_config" "example" {
  provider = grafana.asserts

  name = "ExampleAlert"

  match_labels = {
    alertname = "HighCPUUsage"
    job       = "monitoring"
  }

  alert_labels = {
    severity = "warning"
    team     = "platform"
  }

  duration = "5m"
  silenced = false
}

grafana_asserts_suppressed_assertions_config

Manage Asserts suppressed assertions configurations through the Grafana API.

Arguments

NameTypeRequiredDescription
namestringYesThe name of the suppressed assertions configuration. This field is immutable and forces recreation if changed.
match_labelsmap(string)NoLabels to match for this suppressed assertions configuration. Used to determine which alerts should be suppressed.

Example

terraform 
resource "grafana_asserts_suppressed_assertions_config" "example" {
  provider = grafana.asserts

  name = "ExampleSuppression"

  match_labels = {
    alertname = "TestAlert"
    env       = "development"
  }
}

grafana_asserts_log_config

Manage Asserts log configurations through the Grafana API.

Arguments

NameTypeRequiredDescription
namestringYesThe name of the log configuration. This field is immutable and forces recreation if changed.
prioritynumberYesPriority of the log configuration. Higher priority configurations are evaluated first.
default_configboolYesWhether this is the default configuration. Default configurations cannot be deleted.
data_source_uidstringYesDataSource UID to be queried (for example, a Loki instance).
matchlist(object)NoList of match rules for entity properties. Refer to match block for details.
error_labelstringNoLabel name used to identify error logs.
entity_property_to_log_label_mappingmap(string)NoMapping of entity properties to log labels for correlation.
filter_by_span_idboolNoWhether to filter logs by span ID for distributed tracing correlation.
filter_by_trace_idboolNoWhether to filter logs by trace ID for distributed tracing correlation.

Match block

Each match block supports the following:

NameTypeRequiredDescription
propertystringYesEntity property to match against.
opstringYesOperation to use for matching. One of: EQUALS, NOT_EQUALS, CONTAINS, DOES_NOT_CONTAIN, IS_NULL, IS_NOT_NULL.
valueslist(string)YesValues to match against. Can be empty for IS_NULL and IS_NOT_NULL operations.

grafana_asserts_custom_model_rules

Manage Asserts custom model rules through the Grafana API. This resource allows you to define custom entity models based on Prometheus queries with advanced mapping and enrichment capabilities.

Arguments

NameTypeRequiredDescription
namestringYesThe name of the custom model rules. This field is immutable and forces recreation if changed.
ruleslist(object)YesThe rules configuration containing entity definitions. Refer to rules block for details.

Rules block

Each rules block supports the following:

NameTypeRequiredDescription
entitylist(object)YesList of entity definitions. Refer to entity block for details.

Entity block

Each entity block supports the following:

NameTypeRequiredDescription
typestringYesThe type of the entity (for example, Service, Pod, Namespace).
namestringYesThe name pattern for the entity. Can include pipe-separated alternatives.
defined_bylist(object)YesList of queries that define this entity. Refer to defined_by block for details.
disabledboolNoWhether this entity is disabled. Defaults to false.
enriched_bylist(string)NoList of enrichment sources for the entity.
lookupmap(string)NoLookup mappings for the entity to relate different label names.
scopemap(string)NoScope labels that define the boundaries of this entity type.

defined_by block

Each defined_by block supports the following:

NameTypeRequiredDescription
querystringYesThe Prometheus query that defines this entity.
disabledboolNoWhether this query is disabled. Defaults to false.
label_valuesmap(string)NoLabel value mappings for extracting entity attributes from query results.
literalsmap(string)NoLiteral value mappings for adding static attributes to entities.
metric_valuestringNoMetric value to use from the query result.

Note

When disabled = true is set for a defined_by query, only the query field is used for matching. All other fields in the block are ignored.

Example

terraform 
resource "grafana_asserts_custom_model_rules" "example" {
  provider = grafana.asserts

  name = "example-model"

  rules {
    entity {
      type = "Service"
      name = "service"

      scope = {
        environment = "asserts_env"
        region      = "asserts_site"
      }

      lookup = {
        workload = "deployment"
        pod      = "pod"
      }

      enriched_by = [
        "prometheus_metrics",
        "kubernetes_metadata"
      ]

      defined_by {
        query = "up{service!=''}"
        label_values = {
          service     = "service"
          environment = "asserts_env"
          region      = "asserts_site"
        }
        literals = {
          _source = "service_up"
        }
      }

      defined_by {
        query    = "up{service='test'}"
        disabled = true
      }
    }
  }
}

Apply the Terraform configuration

In a terminal, run the following commands from the directory where all of the configuration files are located.

  1. Initialize a working directory containing Terraform configuration files:

    shell 
    terraform init

  2. Preview the changes that Terraform makes:

    shell 
    terraform plan

  3. Apply the configuration files:

    shell 
    terraform apply

Validation

After you apply the changes in the Terraform configurations, you should be able to verify the following:

  • Notification alerts configurations are created in your Asserts instance and can be viewed in the Asserts UI
  • Suppressed assertions configurations are active and suppressing the specified alerts
  • Log configurations are created and can be used for log drilldown functionality
  • Custom model rules are applied and entities are being discovered according to your defined queries
  • Entity relationships and enrichment are working as expected in the Asserts entity model
  • The configurations are properly scoped to your Grafana Cloud stack

Best practices

When managing Asserts resources with Terraform, consider the following best practices:

Naming conventions

  • Use descriptive names for your notification alerts configurations that clearly indicate their purpose
  • Follow a consistent naming pattern across your organization
  • Include environment or team identifiers in the names when appropriate

Label management

  • Use specific and meaningful labels in match_labels to ensure precise alert filtering
  • Leverage existing label conventions from your monitoring setup
  • Consider using asserts_env and asserts_site labels for multi-environment setups

Silencing strategy

  • Use the silenced parameter for temporary suppression rather than deleting notification alerts configurations
  • Document the reason for silencing in your Terraform configuration comments
  • Regularly review silenced configurations to ensure they’re still needed

Duration configuration

  • Set appropriate duration values based on your alerting requirements
  • Consider the nature of the monitored condition when choosing duration
  • Use consistent duration formats across similar alert types

Custom model rules management

  • Design your entity models to reflect your actual infrastructure and application architecture
  • Use descriptive names for custom model rules that indicate their purpose and scope
  • Start with basic entity definitions and gradually add complexity as needed
  • Test your Prometheus queries independently before using them in model rules

Query design and performance

  • Write efficient Prometheus queries that don’t overload your monitoring system
  • Use specific label filters to reduce the scope of your queries where possible
  • Consider the cardinality implications of your entity definitions
  • Use the disabled flag to temporarily disable problematic queries during debugging

Entity modeling best practices

  • Define clear entity scopes using the scope parameter to organize entities by environment, region, or team
  • Use lookup mappings to establish relationships between different entity types
  • Leverage enriched_by to specify additional data sources for entity enrichment
  • Use meaningful literals to add static metadata that helps with entity identification

Label and attribute management

  • Establish consistent labeling conventions across your infrastructure
  • Map Prometheus labels to entity attributes using clear and descriptive names
  • Use label_values to extract dynamic attributes from your metrics
  • Document the meaning and expected values of custom literals

Conclusion

In this guide, you learned how to use Terraform to manage Grafana Cloud Asserts resources by creating notification alerts configurations, suppressed assertions, log configurations, and custom model rules.

To learn more about managing Grafana Cloud using Terraform, refer to Grafana provider’s documentation.

For more information about Asserts, refer to the Asserts documentation.