Slide 5 of 8

Reliable, repeatable deployments

How Terraform handles existing resources

A reliable pipeline produces the same result no matter how many times you run it, with no duplicate resources and no errors. Terraform ensures this with a state file: a record of the resources it has created. On each run, Terraform decides what to change by comparing three things:

  • Your configuration
  • The state file
  • What actually exists in Grafana

When a live resource no longer matches your code (for example, someone edited an alert threshold in the UI), that difference is called drift. Because of this, you can safely retry failed runs and merge to main repeatedly without creating duplicates.

ScenarioWhat Terraform does
Resource doesn’t existCreates it and records in state
Resource exists and matchesNo changes needed
Resource was edited manuallyDetects drift, restores code-defined version

Singleton resources

Some Grafana resources, like the notification policy tree, exist as a single instance. Terraform handles these correctly: it updates the existing policy tree in place rather than trying to create a second one. This means your pipeline can safely manage the full alerting pipeline, including the notification policy, without conflicts.

Safe retries

The practical benefit of state management: you can rerun your pipeline as many times as you want. Failed runs can be retried safely. Multiple merges to main won’t create duplicate alert rules or contact points. And manual edits in the UI get corrected on the next pipeline run.

Script

A pipeline must be reliable. If you run it multiple times it must produce the same result without errors or duplicates. Terraform handles this automatically through state management.

Terraform tracks which resources it created in a state file. On each run, it compares your configuration to the current state and to what exists in Grafana. This three-way comparison lets Terraform detect drift. If someone manually edited an alert rule in the UI, Terraform will show the difference in its plan and restore the code-defined version on apply.

The practical benefit: you can rerun your pipeline as many times as you want. Failed runs can be retried safely. Multiple merges to main won’t create duplicate resources. And manual edits in the UI get corrected on the next pipeline run.

For resources like notification policies, where only one policy tree exists in Grafana, Terraform’s state management is especially important. It ensures your policy tree is updated in place rather than conflicting with an existing one.

Previous Next