Menu
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Instrument and send databreadcrumb arrow Fleet Managementbreadcrumb arrow Referencebreadcrumb arrow Pipeline HTTP API
Grafana Cloud

Note

Fleet Management is currently in public preview. Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available. For bug reports or questions, fill out our feedback form.

The Pipeline HTTP API

The Grafana Fleet Management Pipeline API allows you to perform CRUD operations on configuration pipelines.

Base URL

The Fleet Management Pipeline API is available at the following base URL:

https://fleet-management-<CLUSTER_NAME>.grafana.net/pipeline.v1.PipelineService/

<CLUSTER_NAME> is the production cluster where your stack is located.

Note

You must have a Grafana Cloud account to find your production cluster.

To find your production cluster, access your Grafana Cloud stack and navigate to Connections > Collector > Configure.

Follow the on-screen instructions to begin an Alloy installation. You don’t need to install Alloy, but you do want to view the installation and set up commands provided by the template. Look for the value of the GCLOUD_FM_URL variable to find your <CLUSTER_NAME>. This is an example of the variable:

GCLOUD_FM_URL="https://fleet-management-prod-001.grafana.net"

In this example, prod-001 is your <CLUSTER_NAME>.

PipelineService

The PipelineService defines the RPCs for managing pipelines.

Method NameRequest TypeResponse TypeDescription
GetPipelineGetPipelineRequestPipelineReturns a pipeline by ID
GetPipelineIDGetPipelineIDRequestGetPipelineIDResponseReturns a pipeline ID by name
ListPipelinesListPipelinesRequestPipelinesReturns all pipelines
CreatePipelineCreatePipelineRequestPipelineCreates a new pipeline and returns it
UpdatePipelineUpdatePipelineRequestPipelineUpdates an existing pipeline and returns it
UpsertPipelineUpsertPipelineRequestPipelineCreates a new pipeline or updates an existing one and returns it
DeletePipelineDeletePipelineRequestDeletePipelineResponseDeletes a pipeline by ID

Endpoints

CreatePipelineRequest

CreatePipelineRequest is the request to create a new pipeline.

FieldTypeLabelDescription
pipelinePipelinerequiredThe pipeline to create
validate_onlybooloptionalIf set, validates the request and previews the response, but doesn’t create the actual resource

DeletePipelineRequest

DeletePipelineRequest is the request to delete a pipeline by its ID.

FieldTypeLabelDescription
idstringrequiredID of the pipeline to delete

DeletePipelineResponse

DeletePipelineResponse is the response to deleting a pipeline. This message is empty and the results of the deletion are defined by the HTTP status code of the response.

GetPipelineRequest

GetPipelineRequest is the request to retrieve a pipeline by its ID.

FieldTypeLabelDescription
idstringrequiredID of the pipeline to get

GetPipelineIDRequest

GetPipelineIDRequest is the request to retrieve a pipeline ID by its name.

FieldTypeLabelDescription
namestringrequiredName of the pipeline to get the ID for

GetPipelineIDResponse

GetPipelineIDResponse is the response to retrieving a pipeline ID.

FieldTypeLabelDescription
idstringID of the pipeline

ListPipelinesRequest

ListPipelinesRequest is the request to get the full list of pipelines, including their contents and matchers.

Pipeline

A Pipeline is a self-contained piece of configuration that can be assigned to collectors based on matchers.

FieldTypeLabelDescription
namestringrequiredName of the pipeline which is the unique identifier for the pipeline
contentsstringrequiredConfiguration contents of the pipeline to be used by collectors
matcherslist(string)Used to match against collectors and assign pipelines to them; follows the syntax of Prometheus Alertmanager matchers
created_atgoogle.protobuf.TimestampoptionalTimestamp when the pipeline was created
updated_atgoogle.protobuf.TimestampoptionalTimestamp when the pipeline was last updated
enabledbooloptionalWhether the pipeline is enabled for collectors
idstringoptionalServer-assigned ID of the pipeline

Pipelines

Pipelines represents a list of pipelines.

FieldTypeLabelDescription
pipelineslist(Pipeline)List of pipelines

UpdatePipelineRequest

UpdatePipelineRequest is the request to update an existing pipeline. This contents supplied in this request replace the existing pipeline contents, so any fields that are not set are removed. If the pipeline does not already exist, this request returns a 404 ‘NOT_FOUND’ error.

FieldTypeLabelDescription
pipelinePipelinerequiredContents of the pipeline to update
validate_onlybooloptionalIf set, validates the request and previews the response, but doesn’t update the actual resource

UpsertPipelineRequest

UpsertPipelineRequest is the request to create a new pipeline or update an existing one. If the pipeline already exists, it is updated and like UpdatePipelineRequest, any fields that are not set are removed.

FieldTypeLabelDescription
pipelinePipelinerequiredPipeline to create or update

Examples

Pipeline API requests and responses require proper escaping of the pipeline contents. For this reason, some use of jq is helpful. In this example, the contents of the pipeline are saved in a file named config.alloy.

alloy 
--- config.alloy ---
prometheus.exporter.self "alloy" { }

prometheus.scrape "alloy" {
  targets    = prometheus.exporter.self.alloy.targets
  forward_to = [prometheus.remote_write.grafanacloud.receiver]

  scrape_interval = "60s"
}

prometheus.remote_write "grafanacloud" {
  // Must match the uniquely-identifiable ID set up in the remotecfg block.
  external_labels = {"collector_id" = constants.hostname}

  endpoint {
    url = <SERVICE_URL>

    basic_auth {
      username      = <USERNAME>
      password_file = <PASSWORD_FILE>
    }
  }
}

To build a payload for the CreatePipeline and UpsertPipeline requests with properly escaped content, the following command is a good starting point. It redirects the content to a create-pipeline.json file for easier handling.

shell 
jq --arg contents "$(cat config.alloy)" \
   --arg name "myname" \
   --argjson matchers '["collector.os=linux", "team!=team-a"]' \
   --argjson enabled true \
   '.pipeline = {name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \
   <<< '{}' > create-pipeline.json

The CreatePipeline response verifies what was created and also lists the unique ID for this pipeline.

shell 
curl -q \
    -d @create-pipeline.json \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/CreatePipeline
{
  "name": "myname",
  "contents": " ... pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "enabled": true,
  "id": "22234"
}

The ListPipelines request takes an empty payload and returns all available pipelines.

shell 
$  curl -q \
    -d '{}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/ListPipelines
{
  "pipelines": [
    {
      "name": "pipeline_1",
      "contents": " ... pipeline contents ...",
      "matchers": [
        "team=\"team-a\""
      ],
      "createdAt": "2024-09-02T12:31:47Z",
      "updatedAt": "2024-09-02T14:08:10Z",
      "enabled": false,
      "id": "22231"
    },
    {
      "name": "pipeline_2",
      "contents": " ... pipeline contents ...",
      "matchers": [
        "collector.os=\"linux\"",
        "team!=\"team-a\""
      ],
      "createdAt": "2024-09-02T16:42:26Z",
      "updatedAt": "2024-09-02T16:42:26Z",
      "enabled": true,
      "id": "22234"
    }
  ]
}

You can use a similar jq command to build an UpdatePipeline request, by adding the unique pipeline ID.

shell 
jq --arg contents "$(cat self.alloy)" \
   --arg id "22234" \
   --arg name "myname" \
   --argjson matchers '["collector.os=linux", "team!=team-a"]' \
   --argjson enabled true \
   '.pipeline = {id: $id, name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \
   <<< '{}' > update-pipeline.json
{
  "name": "myname",
  "contents": " ... new pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "enabled": true,
  "id": "22234"
}

Finally, the GetPipeline and DeletePipeline requests require only an ID.

shell 
curl -q \
    -d '{"id": "22234"}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/GetPipeline
{
  "name": "myname",
  "contents": "... pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "createdAt": "2024-09-02T16:42:26Z",
  "updatedAt": "2024-09-02T16:42:26Z",
  "enabled": true,
  "id": "22234"
}

curl -q \
    -d '{"id": "22234"}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/DeletePipeline
{}