Instrument and send data

Fleet Management

Reference

Pipeline HTTP API

Grafana Cloud

Note
Fleet Management is currently in public preview. Grafana Labs offers limited support, and breaking changes might occur prior to the feature being made generally available. For bug reports or questions, fill out our feedback form.

The Pipeline HTTP API

The Grafana Fleet Management Pipeline API allows you to perform CRUD operations on configuration pipelines.

Base URL

You can find the base URL for the Pipeline API in the Grafana Cloud Fleet Management interface.

In your Grafana Cloud stack, click Connections > Collector > Fleet Management in the left-side menu.
On the Fleet Management interface, switch to the API tab.
Find the URL in the Base URL section. It looks like the following URL, where <CLUSTER_NAME> is the production cluster of your stack:

https://fleet-management-<CLUSTER_NAME>.grafana.net/pipeline.v1.PipelineService/

PipelineService

The PipelineService defines the RPCs for managing pipelines.

Method Name	Request Type	Response Type	Description
GetPipeline	GetPipelineRequest	Pipeline	Returns a pipeline by ID
GetPipelineID	GetPipelineIDRequest	GetPipelineIDResponse	Returns a pipeline ID by name
ListPipelines	ListPipelinesRequest	Pipelines	Returns all pipelines
CreatePipeline	CreatePipelineRequest	Pipeline	Creates a new pipeline and returns it
UpdatePipeline	UpdatePipelineRequest	Pipeline	Updates an existing pipeline and returns it
UpsertPipeline	UpsertPipelineRequest	Pipeline	Creates a new pipeline or updates an existing one and returns it
DeletePipeline	DeletePipelineRequest	DeletePipelineResponse	Deletes a pipeline by ID

Endpoints

CreatePipelineRequest

CreatePipelineRequest is the request to create a new pipeline.

Field	Type	Label	Description
pipeline	Pipeline	required	The pipeline to create
validate_only	bool	optional	If set, validates the request and previews the response, but doesn’t create the actual resource

DeletePipelineRequest

DeletePipelineRequest is the request to delete a pipeline by its ID.

Field	Type	Label	Description
id	string	required	ID of the pipeline to delete

DeletePipelineResponse

DeletePipelineResponse is the response to deleting a pipeline. This message is empty and the results of the deletion are defined by the HTTP status code of the response.

GetPipelineRequest

GetPipelineRequest is the request to retrieve a pipeline by its ID.

Field	Type	Label	Description
id	string	required	ID of the pipeline to get

GetPipelineIDRequest

GetPipelineIDRequest is the request to retrieve a pipeline ID by its name.

Field	Type	Label	Description
name	string	required	Name of the pipeline to get the ID for

GetPipelineIDResponse

GetPipelineIDResponse is the response to retrieving a pipeline ID.

Field	Type	Label	Description
id	string		ID of the pipeline

ListPipelinesRequest

ListPipelinesRequest is the request to get the full list of pipelines, including their contents and matchers.

Pipeline

A Pipeline is a self-contained piece of configuration that can be assigned to collectors based on matchers.

Field	Type	Label	Description
name	string	required	Name of the pipeline which is the unique identifier for the pipeline
contents	string	required	Configuration contents of the pipeline to be used by collectors
matchers	list(string)		Used to match against collectors and assign pipelines to them; follows the syntax of Prometheus Alertmanager matchers
created_at	google.protobuf.Timestamp	optional	Timestamp when the pipeline was created
updated_at	google.protobuf.Timestamp	optional	Timestamp when the pipeline was last updated
enabled	bool	optional	Whether the pipeline is enabled for collectors
id	string	optional	Server-assigned ID of the pipeline

Pipelines

Pipelines represents a list of pipelines.

Field	Type	Label	Description
pipelines	list(Pipeline)		List of pipelines

UpdatePipelineRequest

UpdatePipelineRequest is the request to update an existing pipeline. This contents supplied in this request replace the existing pipeline contents, so any fields that are not set are removed. If the pipeline does not already exist, this request returns a 404 ‘NOT_FOUND’ error.

Field	Type	Label	Description
pipeline	Pipeline	required	Contents of the pipeline to update
validate_only	bool	optional	If set, validates the request and previews the response, but doesn’t update the actual resource

UpsertPipelineRequest

UpsertPipelineRequest is the request to create a new pipeline or update an existing one. If the pipeline already exists, it is updated and like UpdatePipelineRequest, any fields that are not set are removed.

Field	Type	Label	Description
pipeline	Pipeline	required	Pipeline to create or update

Examples

Pipeline API requests and responses require proper escaping of the pipeline contents. For this reason, some use of jq is helpful. In this example, the contents of the pipeline are saved in a file named config.alloy.

--- config.alloy ---
prometheus.exporter.self "alloy" { }

prometheus.scrape "alloy" {
  targets    = prometheus.exporter.self.alloy.targets
  forward_to = [prometheus.remote_write.grafanacloud.receiver]

  scrape_interval = "60s"
}

prometheus.remote_write "grafanacloud" {
  // Must match the uniquely-identifiable ID set up in the remotecfg block.
  external_labels = {"collector_id" = constants.hostname}

  endpoint {
    url = <SERVICE_URL>

    basic_auth {
      username      = <USERNAME>
      password_file = <PASSWORD_FILE>
    }
  }
}

To build a payload for the CreatePipeline and UpsertPipeline requests with properly escaped content, the following command is a good starting point. It redirects the content to a create-pipeline.json file for easier handling.

jq --arg contents "$(cat config.alloy)" \
   --arg name "myname" \
   --argjson matchers '["collector.os=linux", "team!=team-a"]' \
   --argjson enabled true \
   '.pipeline = {name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \
   <<< '{}' > create-pipeline.json

The CreatePipeline response verifies what was created and also lists the unique ID for this pipeline.

curl -q \
    -d @create-pipeline.json \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/CreatePipeline
{
  "name": "myname",
  "contents": " ... pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "enabled": true,
  "id": "22234"
}

The ListPipelines request takes an empty payload and returns all available pipelines.

$  curl -q \
    -d '{}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/ListPipelines
{
  "pipelines": [
    {
      "name": "pipeline_1",
      "contents": " ... pipeline contents ...",
      "matchers": [
        "team=\"team-a\""
      ],
      "createdAt": "2024-09-02T12:31:47Z",
      "updatedAt": "2024-09-02T14:08:10Z",
      "enabled": false,
      "id": "22231"
    },
    {
      "name": "pipeline_2",
      "contents": " ... pipeline contents ...",
      "matchers": [
        "collector.os=\"linux\"",
        "team!=\"team-a\""
      ],
      "createdAt": "2024-09-02T16:42:26Z",
      "updatedAt": "2024-09-02T16:42:26Z",
      "enabled": true,
      "id": "22234"
    }
  ]
}

You can use a similar jq command to build an UpdatePipeline request redirected to update-pipeline.json, by adding the unique pipeline ID.

jq --arg contents "$(cat config.alloy)" \
   --arg id "22234" \
   --arg name "myname" \
   --argjson matchers '["collector.os=linux", "team!=team-a"]' \
   --argjson enabled true \
   '.pipeline = {id: $id, name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \
   <<< '{}' > update-pipeline.json

The UpdatePipeline response verifies what was updated.

curl -q \
    -d @update-pipeline.json \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/UpdatePipeline
{
  "name": "myname",
  "contents": " ... new pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "enabled": true,
  "id": "22234"
}

Finally, the GetPipeline and DeletePipeline requests require only an ID.

curl -q \
    -d '{"id": "22234"}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/GetPipeline
{
  "name": "myname",
  "contents": "... pipeline contents ...",
  "matchers": [
    "collector.os=\"linux\"",
    "team!=\"team-a\""
  ],
  "createdAt": "2024-09-02T16:42:26Z",
  "updatedAt": "2024-09-02T16:42:26Z",
  "enabled": true,
  "id": "22234"
}

curl -q \
    -d '{"id": "22234"}' \
    -u "user:pass" \
    --header "Content-Type: application/json" \
    -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/DeletePipeline
{}

Was this page helpful?

Email docs@grafana.com

Help and support

Community

Feedback

Relevant sources:

Feedback

The Pipeline HTTP API

Base URL

PipelineService

Endpoints

CreatePipelineRequest

DeletePipelineRequest

DeletePipelineResponse

GetPipelineRequest

GetPipelineIDRequest

GetPipelineIDResponse

ListPipelinesRequest

Pipeline

Pipelines

UpdatePipelineRequest

UpsertPipelineRequest

Examples

Was this page helpful?

Related documentation

The Pipeline HTTP API

Base URL

PipelineService

Endpoints

CreatePipelineRequest

DeletePipelineRequest

DeletePipelineResponse

GetPipelineRequest

GetPipelineIDRequest

GetPipelineIDResponse

ListPipelinesRequest

Pipeline

Pipelines

UpdatePipelineRequest

UpsertPipelineRequest

Examples

Was this page helpful?

Related documentation

Related resources from Grafana Labs