This is documentation for the next version of Grafana Alloy Documentation. For the latest stable release, go to the latest version.

Documentationbreadcrumb arrow Grafana Alloybreadcrumb arrow Configurebreadcrumb arrow Clusteringbreadcrumb arrow Distribute workload
Open source

Distribute workload across cluster nodes

After you enable clustering in Alloy, you can configure individual components to distribute their workload across the cluster.

Clustering with target auto-distribution allows a fleet of Alloy instances to dynamically distribute workload and provides high availability and horizontal scalability.

Before you begin

  • Configure clustering in your Alloy installation.
  • Ensure that all clustered Alloy instances have the same configuration file.

Enable clustering in components

To enable workload distribution, add a clustering block with enabled = true inside each component block that should participate:

Alloy 
component.type "example" {
  # ...component settings...

  clustering {
    enabled = true
  }
}

Note

Components don’t automatically participate in clustering. You must explicitly enable clustering in each component that should distribute workload. If cluster mode is enabled but a component doesn’t have clustering { enabled = true }, every node runs that component’s full workload, which causes duplicate scraping, log collection, or profiling.

Components that support clustering

The following components support the clustering block:

Prometheus metrics collection components:

Pyroscope profiling components:

Loki log collection components:

Database observability components:

Example: Distribute Prometheus metrics scrape load

This example shows how to configure prometheus.scrape to distribute scrape targets across cluster nodes.

  1. Add a clustering block to your prometheus.scrape component:

    Alloy 
    prometheus.scrape "default" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [prometheus.remote_write.default.receiver]

  clustering {
    enabled = true
  }
}

  2. Restart or reload Alloy for it to use the new configuration.

  3. Validate that auto-distribution works:

    1. Use the Alloy UI on each node, navigate to the details page for the prometheus.scrape component.

    2. Compare the Debug Info sections between two different nodes to ensure that they don’t scrape the same sets of targets.

Example: Distribute log collection

This example shows how to configure loki.source.kubernetes to distribute log collection across cluster nodes.

Alloy 
loki.source.kubernetes "pods" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [loki.write.default.receiver]

  clustering {
    enabled = true
  }
}

Example: Distribute profiling targets

This example shows how to configure pyroscope.scrape to distribute profiling targets across cluster nodes.

Alloy 
pyroscope.scrape "default" {
  targets    = discovery.kubernetes.pods.targets
  forward_to = [pyroscope.write.default.receiver]

  clustering {
    enabled = true
  }
}

How workload distribution works

When you enable clustering in a component:

  1. All cluster nodes use a consistent hashing algorithm to determine target ownership.
  2. Each node only processes the subset of targets it’s responsible for.
  3. When a node joins or leaves the cluster, targets are automatically redistributed.
  4. Approximately 1/N of targets are redistributed when cluster membership changes. This minimizes disruption.

Note

Target labels must be consistent across all cluster nodes. If target discovery adds node-specific labels, such as constants.hostname, the hashing algorithm can’t consistently determine target ownership.

For more information about clustering concepts, refer to Clustering.