Documentation Grafana Cloud Instrument and send data Fleet Management Manage your fleet Collectors Troubleshoot Internal metrics

Internal metrics

You can use a collector’s internal metrics to identify issues and make decisions about scaling based on resource usage. Visualize these metrics with out-of-the-box dashboards in Grafana Fleet Management.

View health dashboards for a registered collector

The Fleet Management service displays health dashboards in the application interface.

1. In your Grafana Cloud stack, click Connections > Fleet Management.
2. Search, filter, or sort to find the unhealthy collector in your Inventory.
3. Click on the collector to open the details view drawer and view the Health tab.

Fleet Management dashboards

The dashboards available in the Health tab depend on the collector type.

Alloy

The dashboards in the Health tab are based on the Alloy Health integration and default to use metrics scraped by the self_monitoring_metrics pipeline, which is assigned automatically to a registered collector. You can change the data source with the dropdown menu on the Health tab of the details view. If you’ve installed the Alloy Health integration for the collector, a Dashboards button appears in the details view, which takes you to the integration dashboard.

You can view five dashboards for Alloy collectors:

• The Collector dashboard reports metrics from the collector’s component controller.
• The Resources dashboard reports metrics based on the collector’s resource usage.
• The Remote Configuration dashboard reports on the health of remote configuration loading. It includes panels for:
  • The last configuration load status.
  • The configuration load success rate over time.
  • Configuration error logs from the collector’s log stream.
• The Prometheus Components dashboard reports metrics on the performance of Prometheus components in the collector’s configuration, including:
  • prometheus.scrape
  • prometheus.remote_write
• The Cluster Node dashboard reports metrics on the status of communication with peers within a cluster of Alloy instances. Metrics reported on this dashboard begin with cluster_node_*. Clustering must be enabled before Cluster Node metrics are reported.

OpenTelemetry Collector

The Health tab displays a built-in dashboard powered by the collector’s internal telemetry metrics. It defaults to use metrics scraped by a self_monitoring_otelcol pipeline, and a host_metrics_otelcol pipeline if the Collector is deployed outside of Kubernetes. These pipelines are assigned automatically when you register a Collector. You can change the data source with the dropdown menu in the Health tab.

The Health tab of OpenTelemetry Collectors displays a single dashboard with seven panels:

• CPU usage: The rate of CPU time consumed by the collector process. Reported by otelcol_process_cpu_seconds_total.
• Memory (RSS): The resident set size of the collector process. Reported by otelcol_process_memory_rss_bytes.
• Memory (heap Alloc): The total system memory allocated by the collector process. Reported by otelcol_process_runtime_total_sys_memory_bytes.
• Accepted Metrics, Logs, and Spans: The rate of metric points, log records, and spans accepted by the collector’s receivers. Reported by otelcol_receiver_accepted_metric_points_total, otelcol_receiver_accepted_log_records_total, and otelcol_receiver_accepted_spans_total.
• Scrape Errors: The rate of metric points that failed during scraping. Reported by otelcol_scraper_errored_metric_points_total.
• Processor Batches: The rate of data sent by the batch processor. Reported by otelcol_processor_batch_batch_send_size_sum.
• Exported Signals: The rate of metric points, log records, and spans successfully sent by the collector’s exporters. Reported by otelcol_exporter_sent_metric_points_total, otelcol_exporter_sent_log_records_total, and otelcol_exporter_sent_spans_total.