Documentation Grafana Cloud Instrument and send data Fleet Management Manage your fleet Pipelines Troubleshoot

Troubleshoot configuration pipeline issues

This page describes how to troubleshoot issues you might have with your configuration pipelines.

Configuration pipeline was deleted from application

To recover a deleted configuration pipeline, view the pipeline’s history and recreate it:

1. In the Fleet Management application, navigate to the Remote configuration tab.
2. Switch to the Pipelines history view.
3. Search for the deleted configuration pipeline using its name, ID, or matching attributes.
4. Click into the deletion event and copy the configuration pipeline content.
5. Make note of the matching attributes assigned to the pipeline.
6. Switch back to the Remote configuration view and click Create configuration pipeline.
7. Paste the configuration syntax, give it a name, and add the matching attributes.

If you delete one of the autogenerated self-monitoring pipelines, you can also recreate it by deleting the other pipelines named self_monitoring_*. Once the last autogenerated pipeline is deleted, the Grafana Fleet Management service recreates them all.

If you do not want to use the self-monitoring pipelines, switch them off in the Fleet Management interface rather than deleting them.

Configuration pipeline is no longer matched to collector

Configuration pipelines are assigned to collectors based on matching attributes. If a collector is no longer receiving a configuration pipeline, view the pipeline history to see if changes were made to its matching attributes.

1. In the Fleet Management application, navigate to the Remote configuration tab.
2. Switch to the Pipelines history view.
3. Search for the related configuration pipeline using its name or ID.
4. Review the events around the time the issue began, looking for changes in the Matching attributes column.

If no changes were made to the matching attributes of the pipeline, the change must have been made to the collector’s attributes.

Data isn’t flowing as expected

When data isn’t flowing as expected, whether it’s no data, not enough data, or more than expected data, check to see if a change in a configuration pipeline might be the cause.

1. In the Fleet Management application, navigate to the Remote configuration tab.
2. Switch to the Pipelines history view.
3. Search for the related configuration pipeline using its name, ID, or matching attributes.
4. Click into the events that occurred around the time the data issue began.
5. Compare versions of the pipelines using the Diff view until you find the issue.
6. To roll back the change, click the Restore button above the correct version.

If you intentionally make a configuration change to a pipeline, but the data doesn’t reflect the change, check to make sure Alloy was able to fetch and load the new configuration successfully.

1. In the Fleet Management application, navigate to the Inventory tab.
2. Click on the row of a collector that matches to the updated pipeline.
3. In the details view, switch to the Logs tab.
4. Check the Alloy logs for an error message such as msg="failed to fetch remote configuration from the API" service=remotecfg. The log message should also indicate the reason for the failure. For example, a component is missing or an authentication error occurred.

For tips on avoiding configuration rollout problems, refer to Minimize risk by deploying configuration pipelines in stages.

Data isn’t displaying correctly

If your dashboard no longer displays data as you expect, check to see if a configuration change, such as updated relabeling rules, is the cause. Use the Pipelines history feature to review changes to your configuration pipelines. Refer to Data isn’t flowing as expected for guidance on finding pipelines and reverting changes.

Top-level configuration block gives error

Remote configuration does not support top-level configuration blocks. If you want to use a top-level block, you must add the block to the collector’s local configuration file.