Troubleshoot Alloy components

Identify and resolve issues in the Alloy configuration used by the MySQL and PostgreSQL get started guides. These checks help you validate each component without repeating setup. If a check fails, open the corresponding get started article to apply the fix.

What you’ll achieve

In this article, you:

Validate DSN secrets, exporters, and Database Observability components.
Confirm relabeling, scrape jobs, and remote write endpoints.

Before you begin

Ensure you can access your Alloy deployment, view logs, and update configuration. Keep the get started articles open to apply fixes:

Verify Alloy version and stability level

Confirm you run Alloy with the experimental stability level for Database Observability components.

Start Alloy with --stability.level=experimental.

If the stability level is not set, refer to the Run the latest Alloy version section in the get started guides to update the runtime options in your deployment.

Validate DSN secrets for MySQL and PostgreSQL

Ensure your configurations contains valid Data Source Name strings:

For MySQL, the DSN follows user:password@(hostname:port)/.
For PostgreSQL, the DSN follows postgresql://user:password@(hostname:port)/dbname?sslmode=require.

If the secret file is missing or the content is malformed, refer to the MySQL Add the MySQL configuration blocks or PostgreSQL Add the PostgreSQL configuration blocks section to recreate the file with a correct DSN.

Confirm MySQL exporter collectors

Validate that prometheus.exporter.mysql exists and has the Performance Schema collectors enabled.

Ensure enable_collectors includes perf_schema.eventsstatements and perf_schema.eventswaits.

If metrics for statements or waits are missing, refer to the Add the MySQL configuration blocks section to correct the collectors list.

Confirm PostgreSQL exporter collectors and autodiscovery

Validate that prometheus.exporter.postgres exists and exposes statement statistics:

Ensure enabled_collectors includes stat_statements.
Ensure autodiscovery.enabled is true.
If you run on AWS RDS, ensure database_denylist includes rdsadmin.

If exporter targets are empty or metrics lack statement stats, refer to the Add the PostgreSQL configuration blocks section to correct the exporter configuration.

Check Database Observability components

Validate the database_observability.mysql and database_observability.postgres components.

Ensure data_source_name points to the matching secret.
Ensure forward_to targets the appropriate loki.relabel receiver.
Ensure targets reference the matching exporter targets.
If you enabled query_samples, confirm optional settings like disable_query_redaction are intentional.
If you rely on auto-enabling Performance Schema consumers (MySQL), ensure update permissions are granted and allow_update_performance_schema_settings is set.

If some of your database telemetry data is missing, refer to the MySQL Add the MySQL configuration blocks or PostgreSQL Add the PostgreSQL configuration blocks section to check the wiring between secrets, exporters, and Database Observability components.

Validate relabeling rules consistency

Ensure loki.relabel and discovery.relabel rules stay consistent, especially for labels you add.

Keep custom label keys and values consistent in both components.
Ensure the instance and any custom labels are mirrored across rules.

If labels differ between logs and metrics, refer to the MySQL Add the MySQL configuration blocks or PostgreSQL Add the PostgreSQL configuration blocks section to align the relabel rules.

Confirm discovery targets and job label

Verify that discovery.relabel outputs the correct targets and sets the job label.

Use the Database Observability component targets.
Ensure the job label is integrations/db-o11y.

If the scrape job shows no targets or the job label is incorrect, refer to the MySQL Add the MySQL configuration blocks or PostgreSQL Add the PostgreSQL configuration blocks section to correct discovery.relabel rules and targets.

Verify Prometheus scrape configuration

Confirm Prometheus scrapes the output of discovery.relabel and forwards metrics to remote write.

Ensure targets reference discovery.relabel.<name>.output.
Ensure job_name is integrations/db-o11y.
Ensure forward_to points to prometheus.remote_write.metrics_service.receiver.

If metrics do not arrive in Grafana Cloud, refer to the MySQL Add the MySQL configuration blocks or PostgreSQL Add the PostgreSQL configuration blocks section to verify these three settings.

Verify Prometheus remote write endpoint

Confirm the Prometheus remote write endpoint and credentials exist as environment variables.

Ensure GCLOUD_HOSTED_METRICS_URL contains your Grafana Cloud Prometheus URL.
Ensure GCLOUD_HOSTED_METRICS_ID contains your Prometheus instance ID.
Ensure GCLOUD_RW_API_KEY contains a valid Grafana Cloud API token with metrics write permissions.

If remote write fails, refer to the Add Prometheus and Loki write configuration section in the get started guides to validate the values and token permissions.

Verify Loki write endpoint

Confirm the Loki write endpoint and credentials exist as environment variables.

Ensure GCLOUD_HOSTED_LOGS_URL contains your Grafana Cloud Loki URL.
Ensure GCLOUD_HOSTED_LOGS_ID contains your Loki instance ID.
Ensure GCLOUD_RW_API_KEY contains a valid Grafana Cloud API token with logs write permissions.

If logs do not appear in Grafana Cloud, refer to the Add Prometheus and Loki write configuration section in the get started guides to validate the values and token permissions.