GCP Metrics integration for Grafana Cloud

Before you begin

To export metrics from GCP, the Google Stackdriver Exporter is used. That exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials.

Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the roles/monitoring.viewer IAM role contains the required permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Install GCP Metrics integration for Grafana Cloud

1. In your Grafana Cloud stack, click Connections in the left-hand menu.
2. Find GCP Metrics and click its tile to open the integration.
3. Review the prerequisites in the Configuration Details tab and set up Grafana Agent to send GCP Metrics metrics to your Grafana Cloud instance.
4. Click Install to add this integration’s pre-built dashboard to your Grafana Cloud instance, and you can start monitoring your GCP Metrics setup.

Configuration snippets for Grafana Alloy

Advanced mode

These snippets are configured to scrape resource metrics within your GCP projects using the Grafana Agent running locally.

Manually place the following block into your Grafana Alloy configuration file.

Advanced integrations snippets

river Copy

prometheus.exporter.gcp "integrations_gcp" {
        project_ids = [
                "<your-project>",
        ]

        metrics_prefixes = [
                "run.googleapis.com/",
                "cloudfunctions.googleapis.com/",
                "compute.googleapis.com/nat",
                "logging.googleapis.com/billing",
                "logging.googleapis.com/exports",
                "serviceruntime.googleapis.com/quota/",
                "storage.googleapis.com/",
                "pubsub.googleapis.com/subscription",
        ]
}

prometheus.scrape "integrations_gcp" {
    targets    = prometheus.exporter.gcp.integrations_gcp.targets
    forward_to = prometheus.remote_write.metrics_service.receiver
    job_name   = "integrations/gcp"
}

This integration is configured to work with the GCP metrics exporter, which is embedded in the Grafana Agent. Enable the integration by adding the code block in the above section to your agent configuration.

For a full list of configuration values, refer to the component documentation.

Below are the most common arguments:

project_ids

Google Cloud project ID from where we want to scrape metrics from.

metrics_prefixes

Google Monitoring Metric Type prefixes. In configuration, you must specify the prefixes you want to scrape metrics from, each prefix represents a specific GCP resource, potentially followed by the metric types you want to scrape. These can be as targeted or loose as needed. As an example with pubsub metrics:

All metrics:

• pubsub.googleapis.com/

All snapshot specific metrics:

• pubsub.googleapis.com/snapshot

All snapshot specific metrics and a few subscription metrics:

• pubsub.googleapis.com/snapshot
• pubsub.googleapis.com/subscription/num_undelivered_messages
• pubsub.googleapis.com/subscription/oldest_unacked_message_age

extra_filters

Additional filters you can put on top of the metrics described above, to further refine the resources you would like to collect metrics from. The structure for these filters is <targeted_metric_prefix>:<filter_query>. The targeted_metric_prefix is used to ensure the filter is only applied to the metric_prefix(es) where it makes sense.

As an example with pubsub metrics:

metrics_prefixes: pubsub.googleapis.com/snapshot, pubsub.googleapis.com/subscription/num_undelivered_messages

targeted_metric_prefix options would be:

• pubsub.googleapis.com (apply to all defined prefixes)
• pubsub.googleapis.com/snapshot (apply to only snapshot metrics)
• pubsub.googleapis.com/subscription (apply to only subscription metrics)

request_interval

The time range used when querying for metrics. Defaults to 5m when not set.

ingest_delay

Offset for the Google Monitoring Metrics interval into the past by the ingest delay from the metric metadata. When enabled this automatically adjusts the time range used when querying for metrics backwards based on the metadata GCP has published for how long the data can take to be ingested. Defaults to false when not set.

request_offset

Offset for the Google Monitoring Metrics interval into the past. Defaults to 0s when not set.

drop_delegated_projects

Drop metrics from attached projects and fetch project_id only. Defaults to false when not set.

gcp_client_timeout

How long should the collector wait for a result from the API. Defaults to 15s when not set.

Before you begin

To export metrics from GCP, the Google Stackdriver Exporter is used. That exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials.

Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the roles/monitoring.viewer IAM role contains the required permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Install GCP Metrics integration for Grafana Cloud

1. In your Grafana Cloud stack, click Connections in the left-hand menu.
2. Find GCP Metrics and click its tile to open the integration.
3. Review the prerequisites in the Configuration Details tab and set up Grafana Agent to send GCP Metrics metrics to your Grafana Cloud instance.
4. Click Install to add this integration’s pre-built dashboard to your Grafana Cloud instance, and you can start monitoring your GCP Metrics setup.

Post-install configuration for the GCP Metrics integration

This integration is configured to work with the GCP metrics exporter, which is embedded in the Grafana Agent. Enable the integration by adding the code snippet in the above section to your agent configuration file.

Here’s the possible configuration values:

project_ids

Google Cloud project ID from where we want to scrape metrics from.

metrics_prefixes

Google Monitoring Metric Type prefixes. In configuration, you must specify the prefixes you want to scrape metrics from, each prefix represents a specific GCP resource, potentially followed by the metric types you want to scrape. These can be as targeted or loose as needed. As an example with pubsub metrics:

All metrics:

• pubsub.googleapis.com/

All snapshot specific metrics:

• pubsub.googleapis.com/snapshot

All snapshot specific metrics and a few subscription metrics:

• pubsub.googleapis.com/snapshot
• pubsub.googleapis.com/subscription/num_undelivered_messages
• pubsub.googleapis.com/subscription/oldest_unacked_message_age

extra_filters

Additional filters you can put on top of the metrics described above, to further refine the resources you would like to collect metrics from. The structure for these filters is <targeted_metric_prefix>:<filter_query>. The targeted_metric_prefix is used to ensure the filter is only applied to the metric_prefix(es) where it makes sense.

As an example with pubsub metrics:

metrics_prefixes = pubsub.googleapis.com/snapshot, pubsub.googleapis.com/subscription/num_undelivered_messages

targeted_metric_prefix options would be:

• pubsub.googleapis.com (apply to all defined prefixes)
• pubsub.googleapis.com/snapshot (apply to only snapshot metrics)
• pubsub.googleapis.com/subscription (apply to only subscription metrics)

request_interval

The time range used when querying for metrics.

ingest_delay

Offset for the Google Monitoring Metrics interval into the past by the ingest delay from the metric metadata. When enabled this automatically adjusts the time range used when querying for metrics backwards based on the metadata GCP has published for how long the data can take to be ingested.

request_offset

Offset for the Google Monitoring Metrics interval into the past;

drop_delegated_projects

Drop metrics from attached projects and fetch project_id only;

gcp_client_timeout

How long should the collector wait for a result from the API;

Configuration snippets for Grafana Agent

Below integrations, insert the following lines and change the URLs according to your environment:

yaml Copy

  gcp_exporter:
    enabled: true
    project_ids:
    - '<your-project>'
    gcp_client_timeout: 15s
    metrics_prefixes:
      - run.googleapis.com/
      - cloudfunctions.googleapis.com/
      - compute.googleapis.com/nat
      - logging.googleapis.com/billing
      - logging.googleapis.com/exports
      - serviceruntime.googleapis.com/quota/
      - storage.googleapis.com/
      - pubsub.googleapis.com/subscription

Full example configuration for Grafana Agent

Refer to the following Grafana Agent configuration for a complete example that contains all the snippets used for the GCP Metrics integration. This example also includes metrics that are sent to monitor your Grafana Agent instance.

yaml Copy

integrations:
  prometheus_remote_write:
  - basic_auth:
      password: <your_prom_pass>
      username: <your_prom_user>
    url: <your_prom_url>
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_sync_length_seconds_sum|prometheus_target_scrapes_.*|prometheus_target_interval.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
  # Add here any snippet that belongs to the `integrations` section.
  # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  gcp_exporter:
    enabled: true
    project_ids:
    - '<your-project>'
    gcp_client_timeout: 15s
    metrics_prefixes:
      - run.googleapis.com/
      - cloudfunctions.googleapis.com/
      - compute.googleapis.com/nat
      - logging.googleapis.com/billing
      - logging.googleapis.com/exports
      - serviceruntime.googleapis.com/quota/
      - storage.googleapis.com/
      - pubsub.googleapis.com/subscription
logs:
  configs:
  - clients:
    - basic_auth:
        password: <your_loki_pass>
        username: <your_loki_user>
      url: <your_loki_url>
    name: integrations
    positions:
      filename: /tmp/positions.yaml
    scrape_configs:
      # Add here any snippet that belongs to the `logs.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
metrics:
  configs:
  - name: integrations
    remote_write:
    - basic_auth:
        password: <your_prom_pass>
        username: <your_prom_user>
      url: <your_prom_url>
    scrape_configs:
      # Add here any snippet that belongs to the `metrics.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  global:
    scrape_interval: 60s
  wal_directory: /tmp/grafana-agent-wal