Grafana Cloud Monitor infrastructure Kubernetes Monitoring Configure Kubernetes Monitoring Configure with Agent Flow mode

Configure Kubernetes Monitoring with Grafana Agent Flow mode

When you configure with Grafana Agent Flow mode and select all collection choices, the following are installed as part of installation and configuration:

• Grafana Agent Flow mode: Collects metrics, Pod logs, traces, and Kubernetes events. If you choose to enable traces, the Grafana Agent will be configured to receive traces and forward them to Grafana Cloud Traces.
• kube-state-metrics Helm chart: Deploys a kube-state-metrics (KSM) Deployment and Service, along with some other access control objects.
• Node Exporter: Node Exporter runs as a DaemonSet and emits Prometheus metrics for the health and state of the nodes in your cluster.
• OpenCost

Agent Flow mode scrapes the following targets by default:

• cAdvisor (one per Node): cAdvisor is present on each Node in your cluster and emits container resource usage metrics like CPU usage, memory usage, and disk usage.
• kubelet (one per node): kubelet is present on each node and emits metrics specific to the kubelet process like kubelet_running_pods and kubelet_running_container_count.
• kube-state-metrics (one replica, by default): kube-state-metrics runs as a Deployment and Service in your cluster and emits Prometheus metrics that track the state of objects in your cluster, like pods, deployments, and daemonSets.

The default ConfigMap that results from the configuration process creates allowlists. These allowlists are configured to keep a subset of metrics used by the Kubernetes Monitoring dashboards, alerts, and recording rules. You can optionally do any of these with an allowlist:

• Modify it.
• Replace it with a denylist (by using the drop directive).
• Omit it entirely.

To learn more, refer to:

• Use allowlists to manage your configuration
• Reduce Prometheus metrics usage with relabeling
• Reduce Kubernetes metrics usage for other methods of controlling usage

Before you begin

To deploy Kubernetes Monitoring with Agent Flow mode, you need:

• The Admin role to install dashboards and alerts
• A Kubernetes cluster, environment, or fleet you want to monitor
• The kubectl and Helm command-line tools
• Appropriate versions of items related to:
  • kube-state-metrics: Uses client-go to communicate with Clusters. For Kubernetes client-go version compatibility and any other related details, refer to kube-state-metrics.
  • OpenCost: Requires Kubernetes 1.8+ clusters.

Configuration process

Agent Flow mode configuration includes the following steps:

1. Navigate to Kubernetes Monitoring.
2. Begin sending data.
3. Connect your data sources for metrics and logs.
4. Install dashboards and alert rules.
5. Configure with Grafana Agent Flow mode to begin scraping your data.
6. Install integrations.

Navigate to Kubernetes Monitoring

1. At Grafana Cloud Home, click the menu icon.

2. In the main menu, click Observability, then click Kubernetes.

Send data

Click the Start sending data button.

Kubernetes Monitoring displays the Configuration menu item at the Settings tab.

Connect data sources

In the Data sources section of the Settings tab, you can change the default data source that will be used on the Kubernetes Monitoring dashboards and pages.

Install dashboards and alert rules

Within Dashboards, alert rules, and recording rules, click Install dashboards and alert rules to install the out-of-the-box dashboards and alert rules.

Configure Grafana Agent Flow mode

Click the Cluster configuration tab, and complete each section of the instructions.

1. Prerequisites

Make sure you have met the prerequisites required for these configuration steps.

2. Select features and enter cluster information

1. In the Cluster name box, enter the name of your cluster.

2. In the Namespace box, replace default in the with the namespace that you want to monitor in your cluster.

3. Toggle on or off the data you want to collect.

3. Use a Grafana.com Access Policy Token

You can create a new access policy token or use an existing token. Refer to Grafana Cloud Access Policies for more information.

To use an existing token:

1. Click Use an existing token.

2. Paste the token into the Access policy token name box.

To create a new token:

1. Click Create a new token.

2. In the box for Access policy token name, enter the name of your token.

3. Click Create Access policy token.

   The token is generated and appears in the token box. This token is copied into the ConfigMap file.

4. Click the copy icon in the token box to copy the token. Make sure to save it in a secure place. It will not be shown again.

4. Deploy monitoring resources on the Cluster

Deploy the Kubernetes Monitoring Helm chart to the cluster.

1. Copy the command.

2. Paste and run it in your terminal.

5. Done

Optionally, you can click the Cluster status tab to view the cluster status. You will see your data become populated as the system components begin scraping and sending data to Grafana Cloud. This view shows health of the different sources of metrics, logs, and events. If any box shows “Offline”, it can be any of the following:

• The feature was not selected during cluster configuration.
• The system is not running correctly.
• The Grafana Agent was not able to gather that data correctly.

In some cases (specifically, Grafana Agent, Node Exporter, Window Exporter, and OpenCost), the version of that system will be shown.

Install integrations

You can use Grafana integrations to monitor the health and status of services and applications running in your Kubernetes clusters.

To install a Kubernetes integration to begin scraping metrics:

1. Click the Integrations tab.

2. Select the integration for the service you want to monitor.

3. Select your preferred deployment method, either Grafana Agent, Agent Operator, or Agent in Flow mode.

4. Add the provided code snippet to the Grafana Agent ConfigMap, or deploy the provided Operator Object. Make sure to replace any templated fields.

5. Click Install integration. Installed integrations will display a green checkmark on their tile.

If you want to use an integration not currently listed on the page, you can use embedded Agent exporters and integrations by configuring them manually. To learn how, refer to integrations_config.