%3Aquality(90)%2F&w=3840&q=75)
Grafana Alerting: Scale alert routing without scaling complexity using multiple notification policies
Alert routing often starts simple. A team creates a few contact points, adds some label matchers, and builds a notification policy tree that sends each alert to the right destination.
But alerting configurations rarely stay simple.
As an organization grows, its notification policy tree must accommodate more teams, services, and routing requirements. Changes for one team still require editing a global configuration, making ownership less clear and independent provisioning harder. Over time, even small updates can require navigating an increasingly complex routing tree.
With Grafana 13.2, multiple notification policies are now generally available in Grafana-managed alerting. You can create named notification policy trees, assign alert rules to a specific policy, and manage each policy independently.
Existing alert rules continue to use the default notification policy unless you assign them to a named policy, so you can adopt the feature incrementally without migrating every rule at once.
What multiple notification policies enable
Multiple notification policies provide clearer boundaries for organizing, managing, and securing alert routing. With them, you can:
- Split routing logic into smaller, named trees organized around a team, service, domain, or other ownership boundary
- Assign alert rules directly to the policy that should handle their alerts
- Control access using policy-level role-based access control
- Update one policy without affecting the routing configuration or notification state of alerts assigned to another policy
%3Aquality(90)%2F&w=3840&q=75)
The limits of one global notification policy tree
Grafana Alerting’s notification routing model builds on Prometheus Alertmanager, where routing is represented as a single global notification policy tree with a top-level route and nested child routes.
Within that tree, alert labels are matched against routes that determine how alerts are grouped, when notifications are sent, and which contact points receive them.
This model works well when a routing configuration is small or centrally managed. But it introduces challenges as more teams begin sharing the same Grafana stack:
- Every team’s routing logic must fit within the same hierarchy
- Provisioning or updating policies means operating on the entire tree
- Teams need broad access to a shared configuration, even when they own only a small part of it
- Large policy trees become harder to understand, review, and safely change
A common strategy as organizations scale is to create a top-level branch for each team, service, or domain. For example, a global tree might have separate branches for payments, platform, and security alerts.
This provides some logical organization, but the branches are still part of one shared configuration. They have the same lifecycle, the same provisioning boundary, and the same permissions boundary. Updating one team’s branch still means modifying the global tree that contains every other team’s branch.
With multiple notification policies, each team or service can have its own named policy instead of existing only as a branch within the global tree. Each policy can then be provisioned, secured, and managed independently.
The same familiar routing model, with better boundaries
Multiple notification policies let you split that global routing configuration into smaller, named policy trees.
%3Aquality(90)%2F&w=3840&q=75)
Each named policy has its own root policy and child routes. Within the selected tree, routing continues to work as it always has: alert labels are matched against routes, and those routes control grouping, notification timings, and contact point selection.
The main difference is that an alert rule can now select which policy tree should handle its alerts.
For example, an alert rule owned by the payments team can select a policy named “payments.” Alerts produced by that rule are then routed only through the payments tree.
%3Aquality(90)%2F&w=3840&q=75)
The policy selector is part of the alert rule configuration. Rules that do not explicitly select a named policy continue to use the default notification policy, preserving the behavior of existing alerting configurations.
This means you can adopt multiple notification policies incrementally. You can keep the existing tree as your default and introduce additional policies as teams or services are ready to manage their routing independently.
Independently manage and provision each policy
With a single global tree, provisioning was effectively all or nothing. Updating one team’s routes meant submitting or replacing a configuration that also contained every other team’s routing logic.
Named notification policies are individual resources. You can create, retrieve, update, delete, export, and provision one policy without replacing the others.
That makes it possible to align the lifecycle of a policy with the team or service that owns it. A platform team can deploy changes to its “platform” policy without also deploying the “payments” or “security” policies.
Policies are also isolated from one another. Changes to one policy’s routing tree do not alter the routing configuration or notification state of alerts assigned to another policy. Updating the “platform” policy, for example, doesn’t modify or disrupt the alerts being routed through the “payments” policy.
For teams managing alerting as code, this provides a much safer deployment boundary. A Terraform workspace or automation pipeline can own a single named policy instead of requiring ownership of the organization’s complete routing configuration.
Organize routing around how your organization works
There is no single correct way to divide notification policies. The right boundary depends on how your organization owns and operates its systems.
Policies can be organized by:
- Team, such as “payments,” “security,” or “platform”
- Service, such as “checkout-api” or “identity”
- Domain, such as “infrastructure,” “customer-facing,” or “data-platform”
- Environment, when different environments require substantially different routing and ownership
The goal is not necessarily to create a policy for every alert rule. Instead, policies should represent meaningful ownership or lifecycle boundaries.
Smaller trees are easier to review because each one contains only the routing decisions relevant to its purpose. A team can understand its default contact point, escalation paths, grouping configuration, and timing behavior without navigating unrelated routes owned by the rest of the organization.
Delegate ownership with more granular access control
Named policies also provide a natural boundary for more granular role-based access control.
Instead of granting users permission to modify the entire global notification policy tree, administrators can control who is allowed to view, create, update, or delete individual policies. This makes it easier to delegate alert routing to the teams closest to a service while retaining central control over shared or sensitive policies.
Policy-level permissions can be managed directly through the Grafana UI. Administrators can grant teams access to the policies they own without granting broad access to the rest of the organization’s notification routing configuration.
For example, the payments team can manage the “payments” policy without ever receiving permission to change, or even view, the “security” policy. A central observability team can retain control of the default policy and organization-wide fallbacks.
This helps organizations distribute alerting ownership without distributing unrestricted access to the complete notification configuration.
%3Aquality(90)%2F&w=3840&q=75)
Manage policies through the UI, API, or Terraform
You can create and manage named notification policies from the Grafana UI. Navigate to:
Alerts & IRM → Alerting → Notification configuration → Notification policies
From there, create a new policy or select an existing policy to manage its routing tree. When configuring an alert rule, select the notification policy that should receive alerts from that rule.
Named policies are also available through the Grafana Alerting API, allowing automation systems to manage each routing tree as an individual resource.
For Terraform users, the grafana_apps_notifications_routingtree_v1beta1 resource manages named routing trees. The name is supplied through the resource metadata, so separate Terraform resources can own separate policies.
Example:
resource "grafana_apps_notifications_routingtree_v1beta1" "payments" {
metadata {
uid = "payments"
}
spec {
defaults {
receiver = "payments-slack"
group_by = ["alertname", "cluster"]
group_wait = "30s"
group_interval = "5m"
repeat_interval = "4h"
}
routes {
receiver = "payments-oncall"
matchers = [
{
label = "severity"
type = "="
value = "critical"
}
]
}
routes {
receiver = "payments-slack"
matchers = [
{
label = "severity"
type = "="
value = "warn"
}
]
}
}
}Each resource can be managed by the workspace or repository responsible for that policy. The Terraform resource supports nested routes, matchers, grouping, notification timing options, and provenance controls that determine whether the policy remains editable outside Terraform.
Adopt multiple policies incrementally
Multiple notification policies were introduced in Grafana 13.1 behind the alertingMultiplePolicies feature toggle, and they became generally available in Grafana 13.2.
Existing configurations continue to work through the default notification policy, and there is no requirement to immediately divide an existing tree or update every alert rule.
A practical adoption path is:
- Create a named policy for a team, service, or domain.
- Reproduce the relevant routing behavior in the new policy.
- Assign that team’s alert rules to the policy.
- Validate the resulting notification behavior.
- Move management of the policy to the owning team or automation workflow.
Other rules can remain on the default policy until there is a reason to move them.
Alert routing that scales with your organization
A single notification policy tree is simple when an alerting setup is small. At scale, however, it can become a shared configuration bottleneck that is difficult to own, provision, and safely change.
Multiple notification policies preserve Grafana Alerting’s familiar label-based routing model while introducing clearer lifecycle and ownership boundaries. Teams can work with smaller trees, provision policies independently, limit the scope of changes, and apply more granular access controls.
Whether you organize policies by team, service, or domain, the result is alert routing that more closely reflects how your organization actually operates.
Grafana Assistant is the easiest way to get started with metrics, logs, traces, dashboards, and more in Grafana Cloud. We have a generous forever-free tier and plans for every use case. Sign up for free now!
