Grafana Cloud Agent

The Grafana Cloud Agent collects observability data and sends it to Grafana Cloud. Once the agent is deployed to your hosts, it collects and sends Prometheus-style metrics and log data using a pared-down Prometheus collector.

Installing the Grafana Cloud Agent

There are several ways to install and run the agent:

• Use the Kubernetes install script with preconfigured manifests
• Run the agent with Docker
• Run the agent directly on the host
• Modify the example Kubernetes manifests
• Build the agent from source
• Use our production Tanka configs

The full instructions for installing and running the Grafana Cloud Agent are updated frequently as new options are made available.

Note: Users who install and run the agent directly on the host may want the agent to automatically restart when its process is killed or the host is rebooted. To do this, create a systemd service to manage the agent.

Benefits and trade-offs

The Grafana Cloud Agent is designed for easy installation and updates. It uses a subset of Prometheus code features to interact with hosted metrics, specifically:

• Service discovery
• Scraping
• Write ahead log (WAL)
• Remote writing

Along with this, the agent typically uses less memory than scraping metrics using Prometheus.

This optimization of Prometheus for remote write and resource reduction in the agent has resulted in some trade-offs:

• Metrics are pushed rather than pulled
• You can’t query the agent directly; you can only query metrics from the remote write storage
• Recording rules are not supported
• Alerts from the agent are not supported

Typically, Prometheus pulls metrics. The Grafana Cloud Agent reverses this into a push model meaning the agent installed on a monitoring target scraps metrics and pushes metrics to the remote monitoring system versus that remote monitoring system polling (or pulling) metrics from a set of defined targets, as is the case with non-agent Prometheus.

With a pull model, it is straightforward to determine whether a node is available using an up metric with a value of 1 when the target is reachable and 0 when it is not. With the agent’s push model, the up metric has a value of 1 when the agent is running and no value at all when it is not. This distinction is important when monitoring whether your monitoring system is running as expected. To learn more, please see Monitoring Grafana Cloud Agent

While the agent itself can’t use recording rules and alerts, remote write systems like Grafana Cloud support server-side rules and alerts. This means that the reliability of Grafana Cloud Alerts is tied to the reliability of the remote system and that alerts will be delayed at least by the time it takes for samples to reach the remote system.

Integrations

An Integration is the combination of:

• A built-in metrics exporter with sane agent configuration defaults for a given metrics source, like MySQL
• A set of curated dashboards and alerts on Grafana Cloud for a given metrics source, like MySQL

On the Grafana Cloud side, you can see all the integrations available in the Integrations walkthrough.

When the Grafana Cloud team builds Integration for users to install, code is also added to the agent, so that the agent and Grafana Cloud send and receive metrics for the data source. You can’t see which Integrations are supported for your agent version. This makes it important to update your agent version any time you want to use a new Integration you have not previously enabled.

On the host side, where the agent is installed, you may need to edit the agent configuration file to define which data is remote written to Grafana Cloud for use in querying and dashboards. The installation instructions for each individual Integration will tell you if you must do so.