Menu
Grafana Cloud

GCP Metrics integration for Grafana Cloud

Google Cloud Platform (GCP), offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products, such as Google Search, Gmail, Google Drive, and YouTube. Alongside a set of management tools, it provides a series of modular cloud services including computing, data storage, data analytics and machine learning.

The GCP Metrics integration enables you to send metrics from your GCP projects to Grafana Cloud.

This integration includes 1 pre-built dashboard to help monitor and visualize GCP Metrics metrics.

Before you begin

To export metrics from GCP, the Google Stackdriver Exporter is used. That exporter uses the Google Golang Client Library, which offers a variety of ways to provide credentials.

Please refer to the Google Application Default Credentials documentation to see how the credentials can be provided.

If you are using IAM roles, the roles/monitoring.viewer IAM role contains the required permissions. See the Access Control Guide for more information.

If you are still using the legacy Access scopes, the https://www.googleapis.com/auth/monitoring.read scope is required.

Install GCP Metrics integration for Grafana Cloud

  1. In your Grafana Cloud stack, click Connections in the left-hand menu.
  2. Find GCP Metrics and click its tile to open the integration.
  3. Review the prerequisites in the Configuration Details tab and set up Grafana Agent to send GCP Metrics metrics to your Grafana Cloud instance.
  4. Click Install to add this integration’s pre-built dashboard to your Grafana Cloud instance, and you can start monitoring your GCP Metrics setup.

Post-install configuration for the GCP Metrics integration

This integration is configured to work with the GCP metrics exporter, which is embedded in the Grafana Agent. Enable the integration by adding the code snippet in the above section to your agent configuration file.

Here’s the possible configuration values:

project_ids

Google Cloud project ID from where we want to scrape metrics from.

metrics_prefixes

Google Monitoring Metric Type prefixes. In configuration, you must specify the prefixes you want to scrape metrics from, each prefix represents a specific GCP resource, potentially followed by the metric types you want to scrape. These can be as targeted or loose as needed. As an example with pubsub metrics:

All metrics:

  • pubsub.googleapis.com/

All snapshot specific metrics:

  • pubsub.googleapis.com/snapshot

All snapshot specific metrics and a few subscription metrics:

  • pubsub.googleapis.com/snapshot
  • pubsub.googleapis.com/subscription/num_undelivered_messages
  • pubsub.googleapis.com/subscription/oldest_unacked_message_age

extra_filters

Additional filters you can put on top of the metrics described above, to further refine the resources you would like to collect metrics from. The structure for these filters is <targeted_metric_prefix>:<filter_query>. The targeted_metric_prefix is used to ensure the filter is only applied to the metric_prefix(es) where it makes sense.

As an example with pubsub metrics:

metrics_prefixes = pubsub.googleapis.com/snapshot, pubsub.googleapis.com/subscription/num_undelivered_messages

targeted_metric_prefix options would be:

  • pubsub.googleapis.com (apply to all defined prefixes)
  • pubsub.googleapis.com/snapshot (apply to only snapshot metrics)
  • pubsub.googleapis.com/subscription (apply to only subscription metrics)

request_interval

The time range used when querying for metrics.

ingest_delay

Offset for the Google Monitoring Metrics interval into the past by the ingest delay from the metric metadata. When enabled this automatically adjusts the time range used when querying for metrics backwards based on the metadata GCP has published for how long the data can take to be ingested.

request_offset

Offset for the Google Monitoring Metrics interval into the past;

drop_delegated_projects

Drop metrics from attached projects and fetch project_id only;

gcp_client_timeout

How long should the collector wait for a result from the API;

Configuration snippets for Grafana Agent

Below integrations, insert the following lines and change the URLs according to your environment:

yaml
  gcp_exporter:
    enabled: true
    project_ids:
    - '<your-project>'
    gcp_client_timeout: 15s
    metrics_prefixes:
      - run.googleapis.com/
      - cloudfunctions.googleapis.com/
      - compute.googleapis.com/nat
      - logging.googleapis.com/billing
      - logging.googleapis.com/exports
      - serviceruntime.googleapis.com/quota/
      - storage.googleapis.com/
      - pubsub.googleapis.com/subscription

Full example configuration for Grafana Agent

Refer to the following Grafana Agent configuration for a complete example that contains all the snippets used for the GCP Metrics integration. This example also includes metrics that are sent to monitor your Grafana Agent instance.

yaml
integrations:
  prometheus_remote_write:
  - basic_auth:
      password: <your_prom_pass>
      username: <your_prom_user>
    url: <your_prom_url>
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_sync_length_seconds_sum|prometheus_target_scrapes_.*|prometheus_target_interval.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
  # Add here any snippet that belongs to the `integrations` section.
  # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  gcp_exporter:
    enabled: true
    project_ids:
    - '<your-project>'
    gcp_client_timeout: 15s
    metrics_prefixes:
      - run.googleapis.com/
      - cloudfunctions.googleapis.com/
      - compute.googleapis.com/nat
      - logging.googleapis.com/billing
      - logging.googleapis.com/exports
      - serviceruntime.googleapis.com/quota/
      - storage.googleapis.com/
      - pubsub.googleapis.com/subscription
logs:
  configs:
  - clients:
    - basic_auth:
        password: <your_loki_pass>
        username: <your_loki_user>
      url: <your_loki_url>
    name: integrations
    positions:
      filename: /tmp/positions.yaml
    scrape_configs:
      # Add here any snippet that belongs to the `logs.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
metrics:
  configs:
  - name: integrations
    remote_write:
    - basic_auth:
        password: <your_prom_pass>
        username: <your_prom_user>
      url: <your_prom_url>
    scrape_configs:
      # Add here any snippet that belongs to the `metrics.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  global:
    scrape_interval: 60s
  wal_directory: /tmp/grafana-agent-wal

Dashboards

The GCP Metrics integration installs the following dashboards in your Grafana Cloud instance to help monitor your system.

  • GCP Metrics Overview

Overview

Overview

Changelog

md
# 0.0.1 - April 2023

* Initial release

Cost

By connecting your GCP Metrics instance to Grafana Cloud, you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.