--- title: "The Pipeline HTTP API | Grafana Cloud documentation" description: "Reference documentation for the Grafana Fleet Management Pipeline HTTP API, which allows you to perform CRUD operations on configuration pipelines." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # The Pipeline HTTP API The Grafana Fleet Management Pipeline API allows you to perform CRUD operations on configuration pipelines. For a full definition of the Pipeline API, refer to the [`.proto` file](https://github.com/grafana/fleet-management-api/blob/main/api/pipeline/v1/pipeline.proto). ## Find the base URL You can find the base URL for the Pipeline API in the Grafana Cloud Fleet Management interface. 1. In your Grafana Cloud stack, click **Connections** > **Fleet Management** in the left-side menu. 2. On the Fleet Management interface, switch to the **API** tab. 3. Find the URL in the **Base URL** section. It looks like one of the following URLs, depending on the region where your Grafana Cloud stack is hosted. Flat format, where `` is the production cluster of your stack: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none https://fleet-management-.grafana.net/pipeline.v1.PipelineService/ ``` Nested format, where `` is the cloud provider, `` is the cloud provider’s region identifier, and `` distinguishes multiple Grafana Cloud regions hosted in the same cloud provider region: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none https://fleet-management-.--.grafana.net/pipeline.v1.PipelineService/ ``` > Caution > > Always copy the exact URL from your Fleet Management interface instead of constructing it by hand. For details on the URL differences, refer to [Determine Grafana Cloud URLs based on region](/docs/grafana-cloud/security-and-account-management/region-url-formats/). ### Secure connection If you need to secure your API traffic, you can configure a private endpoint in place of the base URL. You can find the service name for endpoint configuration and the private DNS name on the same **API** tab where the base URL is found. For more information, refer to the instructions for your cloud provider: - [AWS PrivateLink](/docs/grafana-cloud/send-data/fleet-management/set-up/connectivity-options/aws-privatelink/) - [Azure Private Link](/docs/grafana-cloud/send-data/fleet-management/set-up/connectivity-options/azure-private-link/) - [GCP Private Service Connect](/docs/grafana-cloud/send-data/fleet-management/set-up/connectivity-options/gcp-private-service-connect/) ## Rate limit You can make a request to the [Fleet Management Tenant API](/docs/grafana-cloud/send-data/fleet-management/api-reference/tenant-api/) to find out the requests-per-second limits applied to your stack. Requests to the Pipeline API are included in the `requests_per_second:api` limit reported by the Tenant API and are limited to 3 requests per second by default. ## PipelineService The `PipelineService` defines the RPCs for managing pipelines. Expand table | Method Name | Request Type | Response Type | Description | |------------------------|-------------------------------|-----------------------------|-----------------------------------------------------------------------| | GetPipeline | GetPipelineRequest | Pipeline | Returns a pipeline by ID | | GetPipelineID | GetPipelineIDRequest | GetPipelineIDResponse | Returns a pipeline ID by name | | ListPipelines | ListPipelinesRequest | Pipelines | Returns all pipelines | | CreatePipeline | CreatePipelineRequest | Pipeline | Creates a new pipeline and returns it | | UpdatePipeline | UpdatePipelineRequest | Pipeline | Updates an existing pipeline and returns it | | UpsertPipeline | UpsertPipelineRequest | Pipeline | Creates a new pipeline or updates an existing one and returns it | | DeletePipeline | DeletePipelineRequest | DeletePipelineResponse | Deletes a pipeline by ID | | BulkDeletePipelines | BulkDeletePipelinesRequest | BulkDeletePipelinesResponse | Deletes multiple pipelines by their IDs | | SyncPipelines | SyncPipelinesRequest | SyncPipelinesResponse | Creates, updates, and deletes multiple pipelines from a common source | | ListPipelinesRevisions | ListPipelinesRevisionsRequest | PipelineRevisions | Returns all pipeline revisions | | ListPipelineRevisions | ListPipelineRevisionsRequest | PipelineRevisions | Returns all pipeline revisions for a single pipeline by ID | | GetPipelineRevision | GetPipelineRevisionRequest | PipelineRevision | Returns a pipeline revision by revision ID | ## Endpoints > Note > > Protobuf field names are defined in `snake_case`. Due to default protojson behavior, field names are converted to `camelCase` in responses. Both `snake_case` and `camelCase` field names are accepted in requests. ### CreatePipelineRequest `CreatePipelineRequest` is the request to create a new pipeline. If no `config_type` is specified in this request, CONFIG\_TYPE\_ALLOY is assigned by default. Expand table | Field | Type | Label | Description | |----------------|----------|----------|-------------------------------------------------------------------------------------------------| | pipeline | Pipeline | required | The pipeline to create | | validate\_only | bool | optional | If set, validates the request and previews the response, but doesn’t create the actual resource | ### DeletePipelineRequest `DeletePipelineRequest` is the request to delete a pipeline by its ID. Requests with an `id` that isn’t a digit-only string fail with a 400 `BAD REQUEST` validation error and the message `id is not a valid integer`. Expand table | Field | Type | Label | Description | |-------|--------|----------|-----------------------------------------------------------------------------| | id | string | required | ID of the pipeline to delete; must be a string of digits (unsigned integer) | ### DeletePipelineResponse `DeletePipelineResponse` is the response to deleting a pipeline. This message is empty and the results of the deletion are defined by the HTTP status code of the response. ### BulkDeletePipelinesRequest `BulkDeletePipelinesRequest` is the request to delete multiple pipelines by their IDs. The operation is atomic: either all pipelines are deleted or none are. If any pipeline in the list does not exist, the entire operation fails with a 404 `NOT FOUND` error and no pipelines are deleted. Expand table | Field | Type | Label | Description | |-------|--------------|----------|-------------------------------------------------------| | ids | list(string) | required | The IDs of the pipelines to delete; must be non-empty | ### BulkDeletePipelinesResponse `BulkDeletePipelinesResponse` is the response to deleting multiple pipelines. This message is empty and the results of the deletion are defined by the HTTP status code of the response. ### GetPipelineRequest `GetPipelineRequest` is the request to retrieve a pipeline by its ID. Expand table | Field | Type | Label | Description | |-------|--------|----------|---------------------------| | id | string | required | ID of the pipeline to get | ### GetPipelineIDRequest `GetPipelineIDRequest` is the request to retrieve a pipeline ID by its name. Expand table | Field | Type | Label | Description | |-------|--------|----------|----------------------------------------| | name | string | required | Name of the pipeline to get the ID for | ### GetPipelineIDResponse `GetPipelineIDResponse` is the response to retrieving a pipeline ID. Expand table | Field | Type | Label | Description | |-------|--------|-------|--------------------| | id | string | | ID of the pipeline | ### ListPipelinesRequest `ListPipelinesRequest` is the request to get the full list of pipelines, including their contents and matchers. You can filter the returned pipelines using local/remote attributes, config type, and enabled state. Expand table | Field | Type | Label | Description | |--------------------|-------------|----------|---------------------------------------------------------------------------------------| | local\_attributes | map(string) | optional | Local attributes used for filtering; combined with `remote_attributes` when filtering | | remote\_attributes | map(string) | optional | Remote attributes used for filtering; combined with `local_attributes` when filtering | | config\_type | ConfigType | optional | Type of config to filter by | | enabled | bool | optional | Whether the pipelines should be enabled or not | ### ConfigType Represents the type of config used by a pipeline. If the `ConfigType` is left unspecified in a `CreatePipeline` request, CONFIG\_TYPE\_ALLOY is assigned by default. Expand table | Value | Description | |---------------------------|------------------------------| | CONFIG\_TYPE\_UNSPECIFIED | Default; do not set manually | | CONFIG\_TYPE\_ALLOY | Alloy configuration | | CONFIG\_TYPE\_OTEL | OpenTelemetry configuration | ### Pipeline A `Pipeline` is a self-contained piece of configuration that can be assigned to collectors based on matchers. Expand table | Field | Type | Label | Description | |--------------|---------------------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | name | string | required | Name of the pipeline which is the unique identifier for the pipeline | | contents | string | required | Configuration contents of the pipeline to be used by collectors | | matchers | list(string) | | Used to match against collectors and assign pipelines to them; follows the syntax of Prometheus [Alertmanager matchers](https://prometheus.io/docs/alerting/latest/configuration/#matcher); 200 character limit per matcher | | created\_at | google.protobuf.Timestamp | optional | Timestamp when the pipeline was created | | updated\_at | google.protobuf.Timestamp | optional | Timestamp when the pipeline was last updated | | enabled | bool | optional | Whether the pipeline is enabled for collectors | | id | string | optional | Server-assigned ID of the pipeline | | source | PipelineSource | optional | Source of the pipeline | | config\_type | ConfigType | optional | Type of configuration (Grafana Alloy or OpenTelemetry Collector); if left unspecified in a `CreatePipeline` request, default value is CONFIG\_TYPE\_ALLOY | ### Pipelines `Pipelines` represents a list of pipelines. Expand table | Field | Type | Label | Description | |-----------|----------------|-------|-------------------| | pipelines | list(Pipeline) | | List of pipelines | ### PipelineSource `PipelineSource` indicates the external source of a pipeline. Although the two fields `type` and `namespace` are `optional`, if `type` is specified, then `namespace` must also be specified. If `type` is set to `UNSPECIFIED`, `namespace` must be an empty string. Requests that don’t follow these rules fail with a 400 `BAD REQUEST` validation error. Expand table | Field | Type | Label | Description | |-----------|----------------------------|----------|---------------------------| | type | PipelineSource\_SourceType | optional | Source of the pipeline | | namespace | string | optional | Namespace of the pipeline | #### PipelineSource\_SourceType Expand table | Value | Description | |---------------------------|--------------------------------------------------------------------------------------| | SOURCE\_TYPE\_UNSPECIFIED | Default value applied when `source` is not specified; do not set this value manually | | SOURCE\_TYPE\_GIT | The pipeline originates from a Git workflow | | SOURCE\_TYPE\_TERRAFORM | The pipeline originates from a Terraform workflow | | SOURCE\_TYPE\_GRAFANA | The pipeline was created by an automated workflow in Grafana Cloud | ### UpdatePipelineRequest `UpdatePipelineRequest` is the request to update an existing pipeline. This contents supplied in this request replace the existing pipeline contents, so any fields that are not set are removed. If the pipeline does not already exist, this request returns a 404 ‘NOT FOUND’ error. Expand table | Field | Type | Label | Description | |----------------|----------|----------|-------------------------------------------------------------------------------------------------| | pipeline | Pipeline | required | Contents of the pipeline to update | | validate\_only | bool | optional | If set, validates the request and previews the response, but doesn’t update the actual resource | ### UpsertPipelineRequest `UpsertPipelineRequest` is the request to create a new pipeline or update an existing one. If the pipeline already exists, it is updated and like `UpdatePipelineRequest`, any fields that are not set are removed. Expand table | Field | Type | Label | Description | |----------------|----------|----------|-------------------------------------------------------------------------------------------------| | pipeline | Pipeline | required | Pipeline to create or update | | validate\_only | bool | optional | If set, validates the request and previews the response, but doesn’t upsert the actual resource | ### ListPipelinesRevisionsRequest `ListPipelinesRevisionsRequest` is the request to get the full list of pipelines revisions, excluding contents. ### ListPipelineRevisionsRequest `ListPipelineRevisionsRequest` is the request to retrieve pipeline revisions by pipeline ID. Expand table | Field | Type | Label | Description | |-------|--------|----------|---------------------------| | id | string | required | ID of the pipeline to get | ### GetPipelineRevisionRequest `GetPipelineRevisionRequest` is the request to retrieve a pipeline revision by revision ID. Expand table | Field | Type | Label | Description | |--------------|--------|----------|------------------------------------| | revision\_id | string | required | ID of the pipeline revision to get | ### PipelineRevision A `PipelineRevision` represents a point in time for a `Pipeline`. Expand table | Field | Type | Label | Description | |--------------|----------------------------|----------|-----------------------------------------------------------------------------| | revision\_id | string | required | Server-assigned ID of the pipeline revision | | snapshot | Pipeline | required | The pipeline at the point in time of the revision | | created\_at | google.protobuf.Timestamp | required | Timestamp when the pipeline revision was created | | operation | PipelineRevision.Operation | required | The operation that was performed on the pipeline which created the revision | #### PipelineRevision.Operation Expand table | Value | Description | |-------------|-------------------------------------------------------| | UNSPECIFIED | Unknown revision operation | | INSERT | The pipeline was created at the time of this revision | | UPDATE | The pipeline was updated at the time of this revision | | DELETE | The pipeline was deleted at the time of this revision | ### PipelineRevisions `PipelineRevisions` represents a list of pipeline revisions. Expand table | Field | Type | Label | Description | |---------------------|------------------------|-------|----------------------------| | pipeline\_revisions | list(PipelineRevision) | | List of pipeline revisions | ### SyncPipelinesRequest `SyncPipelinesRequest` is the request to bulk update pipelines from a common source. If a source pipeline does not exist in the Fleet Management database, it is created. If a source pipeline already exists in the database, it is updated with any changes, the same as if it were updated with an `UpsertPipelineRequest`. If a pipeline exists in the database but not in the source, it is deleted. Expand table | Field | Type | Label | Description | |-----------|----------------|----------|---------------------------| | source | PipelineSource | required | Source of the pipeline(s) | | pipelines | list(Pipeline) | required | List of pipelines | ### SyncPipelinesResponse `SyncPipelinesResponse` is the response to bulk updating pipelines. This message is empty and the results of the sync are defined by the HTTP status code of the response. An HTTP status code of 400 indicates a validation error in the `source` field of the request or in one or more of the pipelines. ## Examples For examples of `SyncPipelines` requests, refer to [Sync configuration pipelines from a Git repository](/docs/grafana-cloud/send-data/fleet-management/manage-fleet/pipelines/sync-pipelines-git/). Pipeline API requests and responses require proper escaping of the pipeline contents. For this reason, some use of `jq` is helpful. In this example, the contents of the pipeline are saved in a file named `config.alloy`. Alloy ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```alloy --- config.alloy --- prometheus.exporter.self "alloy" { } prometheus.scrape "alloy" { targets = prometheus.exporter.self.alloy.targets forward_to = [prometheus.remote_write.grafanacloud.receiver] scrape_interval = "60s" } prometheus.remote_write "grafanacloud" { // Must match the uniquely-identifiable ID set up in the remotecfg block. external_labels = {"collector_id" = constants.hostname} endpoint { url = basic_auth { username = password_file = } } } ``` To build a payload for the `CreatePipeline` and `UpsertPipeline` requests with properly escaped content, the following command is a good starting point. It redirects the content to a `create-pipeline.json` file for easier handling. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell jq --arg contents "$(cat config.alloy)" \ --arg name "myname" \ --argjson matchers '["collector.os=linux", "team!=team-a"]' \ --argjson enabled true \ '.pipeline = {name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \ <<< '{}' > create-pipeline.json ``` The `CreatePipeline` response verifies what was created and also lists the unique ID for this pipeline. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d @create-pipeline.json \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/CreatePipeline { "name": "myname", "contents": " ... pipeline contents ...", "matchers": [ "collector.os=\"linux\"", "team!=\"team-a\"" ], "enabled": true, "id": "22234" } ``` The `ListPipelines` request takes an empty payload and returns all available pipelines. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell $ curl -q \ -d '{}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/ListPipelines { "pipelines": [ { "name": "pipeline_1", "contents": " ... pipeline contents ...", "matchers": [ "team=\"team-a\"" ], "createdAt": "2024-09-02T12:31:47Z", "updatedAt": "2024-09-02T14:08:10Z", "enabled": false, "id": "22231" }, { "name": "pipeline_2", "contents": " ... pipeline contents ...", "matchers": [ "collector.os=\"linux\"", "team!=\"team-a\"" ], "createdAt": "2024-09-02T16:42:26Z", "updatedAt": "2024-09-02T16:42:26Z", "enabled": true, "id": "22234" } ] } ``` You can use a similar `jq` command to build an `UpdatePipeline` request redirected to `update-pipeline.json`, by adding the unique pipeline ID. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell jq --arg contents "$(cat config.alloy)" \ --arg id "22234" \ --arg name "myname" \ --argjson matchers '["collector.os=linux", "team!=team-a"]' \ --argjson enabled true \ '.pipeline = {id: $id, name: $name, contents: $contents, matchers: $matchers, enabled: $enabled}' \ <<< '{}' > update-pipeline.json ``` The `UpdatePipeline` response verifies what was updated. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d @update-pipeline.json \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/UpdatePipeline { "name": "myname", "contents": " ... new pipeline contents ...", "matchers": [ "collector.os=\"linux\"", "team!=\"team-a\"" ], "enabled": true, "id": "22234" } ``` The `GetPipeline` and `DeletePipeline` requests require only an ID for accessing and removing a configuration. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d '{"id": "22234"}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/GetPipeline { "name": "myname", "contents": "... pipeline contents ...", "matchers": [ "collector.os=\"linux\"", "team!=\"team-a\"" ], "createdAt": "2024-09-02T16:42:26Z", "updatedAt": "2024-09-02T16:42:26Z", "enabled": true, "id": "22234" } curl -q \ -d '{"id": "22234"}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/DeletePipeline {} ``` The `BulkDeletePipelines` endpoint allows you to delete multiple pipelines based on their IDs in a single request. This is useful when you need to clean up multiple pipelines at once. The operation is atomic. If any pipeline in the list does not exist, the entire operation fails with a 404 `NOT FOUND` response and no pipelines are deleted. An empty response body with 200 `OK` indicates all pipelines were successfully deleted. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d '{"ids": ["22231", "22234", "22237"]}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/BulkDeletePipelines {} ``` The revision RPCs allow for revisiting the history of changes to pipeline objects for a user. `ListPipelinesRevisions` lists all changes in chronological order alongside the operation that initiated them. For brevity, the response omits the configuration contents of the pipeline in each snapshot. Similarly, `ListPipelineRevisions` allows for viewing the state of a single pipeline ID at every point in time it was changed, along with a full snapshot of the pipeline object. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d '{}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/ListPipelinesRevisions { "pipelineRevisions": [ { "revisionId": "1", "snapshot": { "name": "myname", "matchers": [ "collector.os=\"linux\"", "team!=\"team-a\"" ], "enabled": true, "id": "1" }, "createdAt": "2025-02-14T09:35:54Z", "operation": "INSERT" }, { "revisionId": "2", "snapshot": { "name": "otlp_pipeline", "matchers": [ "collector.os=\"linux\"", "team=\"team-b\"" ], "enabled": true, "id": "2" }, "createdAt": "2025-02-14T09:36:54Z", "operation": "INSERT" }, { "revisionId": "3", "snapshot": { "name": "myname", "matchers": [ "collector.os=\"linux\"", "team=\"team-a\"" ], "enabled": true, "id": "1" }, "createdAt": "2025-02-14T09:37:49Z", "operation": "UPDATE" }, { "revisionId": "4", "snapshot": { "name": "myname", "matchers": [ "collector.os=\"linux\"", "team=\"team-a\"" ], "enabled": true, "id": "1" }, "createdAt": "2025-02-14T09:38:55Z", "operation": "DELETE" }, <...> ] } ``` shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d '{"id": "2"}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/ListPipelineRevisions { "pipelineRevisions": [ { "revisionId": "2", "snapshot": { "name": "myname", "contents": "prometheus.exporter.self \"alloy\" { }\n\nprometheus.scrape \"alloy\" {\n\ttargets = prometheus.exporter.self.alloy.targets\n\tforward_to = [prometheus.remote_write.grafanacloud.receiver]\n\n\tscrape_interval = \"60s\"\n}\n\nprometheus.remote_write \"grafanacloud\" {\n\t// Must match the uniquely-identifiable ID set up in the remotecfg block.\n\texternal_labels = {\"collector_id\" = constants.hostname}\n\n\tendpoint {\n\t\turl = \"http://localhost:8081\"\n\n\t\tbasic_auth {\n\t\t\tusername = \"user\"\n\t\t\tpassword_file = \"/path/to/my/pass\"\n\t\t}\n\t}\n}", "matchers": [ "collector.os=\"linux\"", "team=\"team-a\"" ], "enabled": true, "id": "2" }, "createdAt": "2025-02-14T09:37:49Z", "operation": "UPDATE" } ] } ``` Finally, the `GetPipelineRevision` request returns a singular revision ID, alongside the contents of the pipeline at the given point in time. shell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```shell curl -q \ -d '{"revision_id": "2"}' \ -u "user:pass" \ --header "Content-Type: application/json" \ -X POST https://fleet-management-prod-001.grafana.net/pipeline.v1.PipelineService/GetPipelineRevision { "revisionId": "4", "snapshot": { "name": "myname", "contents": "prometheus.exporter.self \"alloy\" { }\n\nprometheus.scrape \"alloy\" {\n\ttargets = prometheus.exporter.self.alloy.targets\n\tforward_to = [prometheus.remote_write.grafanacloud.receiver]\n\n\tscrape_interval = \"60s\"\n}\n\nprometheus.remote_write \"grafanacloud\" {\n\t// Must match the uniquely-identifiable ID set up in the remotecfg block.\n\texternal_labels = {\"collector_id\" = constants.hostname}\n\n\tendpoint {\n\t\turl = \"http://localhost:8081\"\n\n\t\tbasic_auth {\n\t\t\tusername = \"user\"\n\t\t\tpassword_file = \"/path/to/my/pass\"\n\t\t}\n\t}\n}", "matchers": [ "collector.os=\"linux\"", "team=\"team-a\"" ], "enabled": true, "id": "1" }, "createdAt": "2025-02-14T09:38:55Z", "operation": "DELETE" } ```