Configure Kubernetes Monitoring with Grafana Agent Flow mode
Note: Grafana Agent Flow mode is recommended for use. However, you can also configure with Grafana Agent or Grafana Agent Operator manually.
When you configure with Grafana Agent Flow mode and select all collection choices, the following are installed as part of installation and configuration:
- Grafana Agent Flow mode: Collects metrics, Pod logs, traces, and Kubernetes events. If you choose to enable traces, the Grafana Agent will be configured to receive traces and forward them to Grafana Cloud Traces.
- kube-state-metrics Helm chart: Deploys a kube-state-metrics (KSM) Deployment and Service, along with some other access control objects.
- Node Exporter: Node Exporter runs as a DaemonSet and emits Prometheus metrics for the health and state of the nodes in your cluster.
Note: You can choose to toggle off the collection of metrics, logs, events, traces, or costs during the configuration process.
Agent Flow mode scrapes the following targets by default:
- cAdvisor (one per Node): cAdvisor is present on each Node in your cluster and emits container resource usage metrics like CPU usage, memory usage, and disk usage.
- kubelet (one per node): kubelet is present on each node and emits metrics specific to the kubelet process like
- kube-state-metrics (one replica, by default): kube-state-metrics runs as a Deployment and Service in your cluster and emits Prometheus metrics that track the state of objects in your cluster, like pods, deployments, and daemonSets.
The default ConfigMap that results from the configuration process creates allowlists. These allowlists are configured to keep a subset of metrics used by the Kubernetes Monitoring dashboards, alerts, and recording rules. You can optionally do any of these with an allowlist:
- Modify it.
- Replace it with a denylist (by using the
- Omit it entirely.
To learn more, refer to:
- Use allowlists to manage your configuration
- Reduce Prometheus metrics usage with relabeling
- Reduce Kubernetes metrics usage for other methods of controlling usage
Before you begin
To deploy Kubernetes Monitoring with Agent Flow mode, you need:
- The Admin role to install dashboards and alerts
- A Kubernetes cluster, environment, or fleet you want to monitor
kubectland Helm command-line tools
- Appropriate versions of items related to:
Agent Flow mode configuration includes the following steps:
- Navigate to Kubernetes Monitoring.
- Begin sending data.
- Connect your data sources for metrics and logs.
- Install dashboards and alert rules.
- Configure with Grafana Agent Flow mode to begin scraping your data.
- Install integrations.
Navigate to Kubernetes Monitoring
At Grafana Cloud Home, click the menu icon.
In the main menu, click Observability, then click Kubernetes.
Click the Start sending data button.
Kubernetes Monitoring displays the Configuration menu item at the Settings tab.
Connect data sources
In the Data sources section of the Settings tab, you can change the default data source that will be used on the Kubernetes Monitoring dashboards and pages.
Install dashboards and alert rules
Within Dashboards, alert rules, and recording rules, click Install dashboards and alert rules to install the out-of-the-box dashboards and alert rules.
Configure Grafana Agent Flow mode
Click the Cluster configuration tab, and complete each section of the instructions.
Make sure you have met the prerequisites required for these configuration steps.
2. Select features and enter cluster information
In the Cluster name box, enter the name of your cluster.
In the Namespace box, replace
defaultin the with the namespace that you want to monitor in your cluster.
Toggle on or off the data you want to collect.
3. Use a Grafana.com Access Policy Token
You can create a new access policy token or use an existing token. Refer to Grafana Cloud Access Policies for more information.
To use an existing token:
Click Use an existing token.
Paste the token into the Access policy token name box.
To create a new token:
Click Create a new token.
In the box for Access policy token name, enter the name of your token.
Click Create Access policy token.
The token is generated and appears in the token box. This token is copied into the ConfigMap file.
Click the copy icon in the token box to copy the token. Make sure to save it in a secure place. It will not be shown again.
4. Deploy monitoring resources on the Cluster
Deploy the Kubernetes Monitoring Helm chart to the cluster.
Copy the command.
Paste and run it in your terminal.
Optionally, you can click the Cluster status tab to view the cluster status. You will see your data become populated as the system components begin scraping and sending data to Grafana Cloud. This view shows health of the different sources of metrics, logs, and events. If any box shows “Offline”, it can be any of the following:
- The feature was not selected during cluster configuration.
- The system is not running correctly.
- The Grafana Agent was not able to gather that data correctly.
In some cases (specifically, Grafana Agent, Node Exporter, Window Exporter, and OpenCost), the version of that system will be shown.
You can use Grafana integrations to monitor the health and status of services and applications running in your Kubernetes clusters.
To install a Kubernetes integration to begin scraping metrics:
Click the Integrations tab.
Select the integration for the service you want to monitor.
Select your preferred deployment method, either Grafana Agent, Agent Operator, or Agent in Flow mode.
Add the provided code snippet to the Grafana Agent ConfigMap, or deploy the provided Operator Object. Make sure to replace any templated fields.
Click Install integration. Installed integrations will display a green checkmark on their tile.
If you want to use an integration not currently listed on the page, you can use embedded Agent exporters and integrations by configuring them manually. To learn how, refer to integrations_config.