Documentation Grafana Cloud Alerts and IRM IRM Manage On-call schedules Schedules as code

Schedules as code

This topic explains how to manage on-call schedules in Grafana Incident Response Management (IRM) using Infrastructure as Code (IaC).

Why use Infrastructure as Code for schedules?

Instead of manually creating and updating on-call schedules through the UI, you can define them as code for better reliability and team collaboration.

• Automation & efficiency: Eliminate manual schedule setup and reduce human error.
• Consistency & standardization: Standardize rotation patterns across multiple teams.
• Version Control & collaboration: Track schedule changes in Git and enable team reviews and approvals for schedule modifications.
• DevOps integration: Integrate schedule updates into your existing CI/CD workflows.
• Scale with confidence: Manage dozens of teams and replicate successful schedule patterns across your organization.

Choose your approach

Grafana IRM supports two primary methods for managing schedules as code:

1. Terraform with the Grafana provider - Ideal for teams already using Terraform or wanting declarative configuration.
2. Grafana IRM API directly - Ideal for custom integrations or dynamic schedule management.

Manage on-call schedules with Terraform

Terraform allows you to define on-call schedules as code, ensuring reproducibility and version control. Using the Grafana Terraform provider, you can create and manage on-call schedules within Grafana IRM.

Before you begin

You need:

• A Grafana Cloud account using IRM
• A Grafana Service Account token or a Grafana IRM API token with appropriate permissions
• Terraform installed (Install Terraform)
• Grafana provider configured (Terraform Provider Documentation)

For initial Terraform setup, refer to Infrastructure as Code.

Understand on-call schedules and shifts in Terraform

Before creating schedules with Terraform, it’s important to understand the key concepts and components.

• Schedule: A container that holds one or more shifts and defines the overall on-call calendar.
• Shift: The actual rotation pattern that determines who is on-call and when.

Key schedule configuration parameters

When configuring schedules in Terraform, pay attention to these parameters:

• name: A descriptive name for your schedule (e.g., “Backend Team Primary Rotation”)
• type: Schedule type - use "calendar" to define shifts via Terraform
• time_zone: The time zone for the schedule (e.g., “UTC”, “America/New_York”, “Europe/London”)
• shifts: List of shift IDs that belong to this schedule
• team_id: Associates the schedule with a specific team for better organization
• slack: Slack workspace integration settings for notifications

For a detailed list of available resources, refer to the Grafana oncall schedule terraform Schema.

Key shift configuration parameters

When configuring shifts in Terraform, pay attention to:

• name: A descriptive name for your shift
• start: The date and time when the shift pattern begins
• duration: How long each person/group stays on-call
• frequency: How often the pattern repeats (daily, weekly, monthly)
• by_day: Which days of the week the shift applies to
• level: Indicates the shift priority, useful when overriding shifts
• type: The type of shift (rolling_users or single_event)

For a detailed list of available resources, refer to the Grafana oncall shift terraform Schema.

Shift types

Grafana IRM supports the following shift types:

• rolling_users: The most common and flexible shift type for creating recurring rotations where users alternate being on-call. Use this for weekly rotations, daily handoffs, or any pattern where the same users rotate in a predictable order.

• single_event: Creates a one-time shift for a specific date and time. Useful for covering special events, holidays, or temporary coverage.

• recurrent_event: Deprecated - Creates recurring shifts based on specific dates and times. Use rolling_users instead for new implementations, as it provides better flexibility and is actively maintained.

Rolling users shift patterns

The rolling_users shift type offers the most flexibility for on-call rotations:

• Individual rotations: Each user takes turns being on-call alone: [[user1], [user2], [user3]]
• Co-primary rotations: Multiple users share on-call duties simultaneously: [[user1, user2], [user3, user4]]
• Mixed patterns: Combine individual and group rotations: [[user1], [user2, user3], [user4]]

Example: Creating an on-call schedule with Terraform

The following example shows a basic on-call schedule configuration:

terraform Copy

provider "grafana" {
  alias = "oncall"

  url        = "<Stack-URL>"
  auth       = "<Service-account-token>"
  oncall_url = "<OnCall-URL>"
}

resource "grafana_oncall_schedule" "example_schedule" {
  provider = grafana.oncall

  name      = "Primary On-Call Schedule"
  type      = "calendar"
  time_zone = "UTC"
}

Set up an on-call rotation via Terraform

The following example demonstrates how to set up a weekly rotation where two users alternate on-call duties.

Define users

First, reference the users who will participate in the rotation:

terraform Copy

data "grafana_oncall_user" "user_one" {
  provider = grafana.oncall
  username = "user_one"
}

data "grafana_oncall_user" "user_two" {
  provider = grafana.oncall
  username = "user_two"
}

Create an on-call shift

Next, define the shift pattern:

terraform Copy

resource "grafana_oncall_on_call_shift" "week_shift" {
  provider   = grafana.oncall
  name       = "Week shift"
  type       = "rolling_users"
  start      = "2025-03-01T00:00:00"
  duration   = 60 * 60 * 24  # 24 hours in seconds
  frequency  = "weekly"
  by_day     = ["MO", "TU", "WE", "TH", "FR", "SA", "SU"]
  week_start = "MO"
  rolling_users = [
    [data.grafana_oncall_user.user_one.id],
    [data.grafana_oncall_user.user_two.id]
  ]
  time_zone = "UTC"
}

For all available parameters, refer to the Grafana Terraform provider reference.

Connect the shift to a schedule

Create a schedule and associate it with the shift:

terraform Copy

resource "grafana_oncall_schedule" "primary" {
  provider  = grafana.oncall

  name      = "Primary"
  type      = "calendar"
  time_zone = "UTC"
  shifts    = [
    grafana_oncall_on_call_shift.week_shift.id
  ]
}

Add the schedule to an escalation chain

Finally, add the schedule to an escalation chain to use it for incident notifications:

terraform Copy

resource "grafana_oncall_escalation" "notify_schedule" {
  provider                     = grafana.oncall
  escalation_chain_id          = grafana_oncall_escalation_chain.default.id
  type                         = "notify_on_call_from_schedule"
  notify_on_call_from_schedule = grafana_oncall_schedule.primary.id
  position                     = 0
}

After applying your configuration with terraform apply, you can view your schedule in Grafana IRM under the Schedules tab or export it as an iCal link.

Manage on-call schedules with the API

The Grafana IRM API provides granular control over on-call schedules, enabling dynamic updates and integrations with other systems.

API endpoints

The On-Call Schedules API supports the following operations:

• List schedules: GET /api/v1/schedules/
• Create a schedule: POST /api/v1/schedules/
• Update a schedule: PUT /api/v1/schedules/{schedule_id}
• Delete a schedule: DELETE /api/v1/schedules/{schedule_id}

For complete API documentation, refer to the Grafana IRM API Reference: Schedules.

Example: Creating an on-call schedule via API

The following example shows how to create a schedule using a curl command:

Bash Copy

curl -X POST "https://<OnCall-API-URL>/api/v1/schedules/" \
-H "Authorization: Bearer $GRAFANA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "name": "Primary On-Call Schedule",
  "team_id": "your-team-id",
  "type": "calendar",
  "time_zone": "UTC"
}'

Time zone considerations

When working with schedules as code, time zones are a critical consideration:

• Always specify the time zone for each rotation to avoid scheduling confusion
• Use consistent time zone formats across your configuration
• Consider the global distribution of your team when setting up rotations
• Test schedules across time zone boundaries to ensure proper handoffs
• Be aware of daylight saving time transitions when scheduling long-term rotations

Best practices for schedules as code

Follow these best practices when managing your on-call schedules as code:

• Use source control: Store Terraform configurations and API scripts in a Git repository to track changes
• Automate deployments: Use CI/CD tools like GitHub Actions or Terraform Cloud for managing schedules across environments
• Apply role-based access control (RBAC): Restrict who can modify schedules via API keys and IAM policies to prevent unauthorized changes
• Test before applying: Use terraform plan or API dry-runs to validate configurations before deploying to production
• Document your approach: Maintain documentation about your schedule structure and rotation patterns
• Implement monitoring: Set up alerts for schedule configuration changes or failures
• Use variables and modules: Leverage Terraform variables and modules to create reusable schedule patterns

Additional resources

• Grafana IRM API Reference: Schedules
• Terraform Provider for Grafana
• Getting Started with Terraform and Grafana IRM