Grafana Cloud Grafana Incident Grafana Incident APIs Grafana Incident JSON/HTTP RPC API

Note: This feature is currently experimental or under active development. Some details may change when the feature is released.

Grafana Incident JSON/HTTP RPC API

You can programmatically control Grafana Incident via the Grafana Incident JSON/HTTP RPC API.

Grafana Incident provides a simple JSON/HTTP RPC request/reply API, that exposes a series of useful methods for you to call remotely from your own code.

The endpoint is available at the following URL:

https://your-stack.grafana.net/api/plugins/grafana-incident-app/resources/api/experimental

Note: You will need to replace your-stack.grafana.net with your own instance.

You can explore the entire API surface in the Grafana Incident JSON/HTTP RPC API Reference (experimental) documentation or continue reading for an introduction.

Use cases

You can use the Grafana Incident JSON/HTTP RPC API along with the Outgoing Webhooks integration to solve a wide range of integration use cases.

Some examples include:

• Trigger a ServiceNow workflow whenever an incident is declared
• Automatically add some common tasks whenever a specific label is added
• Send the Incident summary in an email when it is resolved
• Send a Slack DM to the Security Team if somebody adds the security label

How it works

To make calls into the API, you can use one of the official client libraries or construct your own HTTP requests.

You will need to use a Service account token in the Authorization header in order to authenticate your request.

An RPC API is like calling remote methods. For every request you make, you get a response.

Examples

This section provides some real-world examples of how you will typically interact with the Grafana Incident JSON/HTTP RPC API.

Although the examples are in Go, your code will likely look very similar regardless of your language choice.

Declaring an incident

To programmatically declare an incident in Go, using the Grafana Incident Go client library.

import (
  "fmt"
  "os"
  "github.com/grafana/incident-api/go/incident"
)

func main() {
  // create a client, and the services you need
  serviceAccountToken := os.Getenv("SERVICE_ACCOUNT_TOKEN")
  client := incident.NewClient(
    "https://your-stack.grafana.net/api/plugins/grafana-incident-app/resources/api/experimental",
    serviceAccountToken,
  )
  incidentsService := incident.NewIncidentsService(client)

  // declare an incident
  createIncidentResp, err := incidentsService.CreateIncident(ctx, incident.CreateIncidentRequest{
    Title: "short description explaining what's going wrong",
  })
  if err != nil {
    // if something goes wrong, the error will help you
    return fmt.Errorf("create incident: %w", err)
  }

  // success, get the details from the createIncidentResp object
  incidentID := createIncidentResp.Incident.IncidentID
  fmt.Println("declared Incident", createIncidentResp.Incident.IncidentID)

Automatically add a task

To add a task to the above Incident, we can use the *incident.TasksService with the same client:

tasksService := incident.NewTasksService(client)
addTaskResp, err := tasksService.AddTask(ctx, incident.AddTaskRequest{
  IncidentID: incidentID,
  Text:       "Consider tweeting to let people know",
})
if err != nil {
	// if something goes wrong, the error will help you
	return fmt.Errorf("add task: %w", err)
}
// success, print the result
fmt.Println("added task %s to incident %s", addTaskResp.Task.TaskID, incidentID)

Post a Slack message if a specific label is added

By using webhooks when an incident is declared or updated,

Note It is possible for webhooks to fire multiple times as the Incident changes. It is up to you to make sure you don’t spam people.

getIncidentResp, err := incidentsService.GetIncident(ctx, incident.GetIncidentRequest{
  IncidentID: incidentID,
})
if err != nil {
  return fmt.Errorf("get incident %s: %w", incidentID, err)
}

for _, item := range getIncidentResp.Incident.Labels {
  if item.Label == "specific-label" {
    postToSlack(ctx, "An incident has been flagged with the security label.")
    return
  }
}

You would write your own postToSlack method to interact with Slack’s API.