--- title: "Configuration steps for Kubernetes Monitoring with Helm chart | Grafana Cloud documentation" description: "Steps to configure Kubernetes Monitoring with Helm chart" --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Configuration steps for Kubernetes Monitoring with Helm chart Complete the configuration process on the **Cluster configuration** tab. The wizard guides you through five steps: **Backend and distribution**, **Access token**, **Choose setup**, **Monitoring**, and **Deployment**. Complete each step, then click **Next**. Give it a try using Grafana Play With Grafana Play, you can explore and see how it works, learning from practical examples to accelerate your development. This feature can be seen on [this **Configuration** page](https://play.grafana.org/a/grafana-k8s-app/configuration?from=now-1h&to=now). [Try it](https://play.grafana.org/a/grafana-k8s-app/configuration?from=now-1h&to=now) ## Before you begin Make sure you have met the prerequisites required for these configuration steps. > Note > > Ensure that you are familiar with the [components installed by the Helm chart](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/helm-chart-config/helm-chart/#packages-installed-with-helm-chart) and how they relate to switching on or off the configuration choices available. To deploy Kubernetes Monitoring with the Helm chart, you need: - A Grafana Cloud account - The Admin role to install alerts - To have completed the [activation of the app](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/activate/) - A Kubernetes Cluster, environment, or fleet you want to monitor - Helm installed - The `kubectl` and [Helm](https://helm.sh/) command-line tools - Appropriate versions of items related to: - kube-state-metrics: Uses `client-go` to communicate with Clusters. For Kubernetes client-go version compatibility and any other related details, refer to [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics#versioning). - OpenCost: Requires [Kubernetes 1.8+](https://github.com/opencost/opencost#getting-started) clusters. - Visualizations for storage: Require [Helm chart](https://github.com/grafana/k8s-monitoring-helm/tree/main/charts/k8s-monitoring) [release](https://github.com/grafana/k8s-monitoring-helm/releases) v1.5.1 or later ## Backend and distribution Install the required backend rules and choose the platform for your Cluster. > Note > > The backend installation installs rules that are *required* for Kubernetes Monitoring to function properly. Recording rules are the source of the workload data in Kubernetes Monitoring. If you aren’t seeing the workload data, the most likely cause is that the recording rules and alert rules haven’t been installed. 1. Under **Backend Installation**, click **Install** to install the required, preconfigured [alert rules](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/manage-configuration/#alerting-rules) and [recording rules](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/#recording-rules) to your Grafana instance. When they’re installed, the status shows **Alert rules and recording rules installed**. 2. Under **Select your Kubernetes platform**, choose your platform: - **Kubernetes**, for EKS on EC2, GKE, and similar distributions - **Azure AKS** - **GKE Autopilot** - **EKS on Fargate** - **IBM Cloud** - **OpenShift** 3. Click **Next**. ## Access token An access policy token is required for Grafana Alloy to send metrics and logs to Grafana Cloud. Choose how the Helm chart authenticates to Grafana Cloud. You can create a new access policy token, use an existing token, or point the chart at a stored Kubernetes Secret that already exists in your Cluster. Refer to [Grafana Cloud Access Policies](/docs/grafana-cloud/account-management/authentication-and-permissions/access-policies/) for more information. To create a new token: 1. Click **Create a new token**. 2. In the **Access policy token name** box, enter a name. This name helps you view and revoke the token in Grafana.com later. 3. In the **Expiration date** box, select or enter an expiration in days, or keep **No expiry**. The token uses the predefined scopes `metrics:read` and `set:alloy-data-write`. 4. Click **Create token**. To use an existing token: 1. Click **Use an existing token**. 2. Paste your token into the box. To use a stored Kubernetes Secret: 1. Click **Use a stored Kubernetes Secret**. 2. In the **Secret name** box, enter the name of the Secret that already exists in your Cluster. 3. In the **Namespace** box, keep the installation namespace or enter the namespace that holds the Secret. 4. Optionally, expand the advanced options to change the **Username key** and **Password key** if your Secret doesn’t use the default keys `username` and `password`. The Secret must hold your Grafana Cloud instance ID as the username and an access policy token as the password, with the `metrics:read` and `set:alloy-data-write` scopes. The configuration page provides a `kubectl` command, prefilled with your instance ID, that you can copy to create the Secret. Because the chart reads the token from the Secret at runtime, your token isn’t stored in Grafana and doesn’t appear in the generated Helm command. 5. Click **Next**. ## Choose setup Select a preconfigured Alloy deployment based on the level of observability you need. Each option determines what telemetry you can collect now and what you can enable later without reinstalling. Expand **Learn what this deployment includes** on any option to see its details. Your choice sets the collector presets in the generated Helm values. - **Scalable - High performance** (Recommended): Runs one Alloy collector on every Node (a DaemonSet) plus a separate clustered backend (a Deployment) for cluster-wide aggregation. The per-Node collectors use the `xlarge` size and vertically autoscale between 1 and 4 CPU cores and 2 to 8 GiB of memory. The backend horizontally autoscales from 1 to 5 replicas when CPU passes 75% or memory passes 80%, and it buffers to persistent storage (PVC). Best for large-scale production environments, high-traffic systems, and mission-critical services. - **Standard - Balanced cluster setup**: Runs one Alloy collector on every Node (a DaemonSet) plus a clustered backend (a Deployment) for aggregation, both at the fixed `large` size (0.5 to 2 CPU cores and 1 to 2 GiB of memory per instance). It doesn’t autoscale and has limited buffering. Best for most production workloads, growing teams, and medium to large Clusters. - **Lightweight - Quick start**: Runs a single Alloy collector on every Node (a DaemonSet) at the `large` size, with no separate backend, no cluster-wide aggregation, and no persistent storage. Best for smaller Clusters, early-stage environments, and cost-conscious setups. Your tier selection also seeds the default monitoring options on the **Monitoring** step, which you can still change. **Scalable** and **Standard** enable the recommended signals, which is everything except Node logs and trace forwarding. **Lightweight** enables only Cluster metrics and Cluster events. Click **Next**. ## Monitoring On the **Remote monitoring and service discovery** step, enable the remote monitoring features and managed pipelines you want from Grafana Cloud. You can change these selections later in Instrumentation Hub without a Helm upgrade. Kubernetes monitoring is required by this flow, and auto-discovery of services is optional. 1. Select **Enable Kubernetes monitoring** to instrument your Cluster with the recommended signals for most production Clusters. You can customize the signals later in Instrumentation Hub. > Note > > After you activate Kubernetes monitoring, billing begins based on your host and container hours. 2. Optionally, select **Auto-discover services on my cluster** to create a managed discovery pipeline in Fleet Management. No application telemetry is collected unless you enable instrumentation. To change this, open the card’s menu, select **Managed discovery pipeline settings**, then choose **Recommended** or **Not recommended**. 3. Click **Next**. To customize the signals collected for Kubernetes monitoring, open the **Enable Kubernetes monitoring** card’s menu and select one of the tuning options. Switch each signal on or off, then click **Save**. > Note > > These options are independent of each other. For example, disabling cost or energy metrics does not disable any other option. Refer to additional information for each option by following the links and review [Manage your Kubernetes configuration](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/manage-configuration/#cluster-events). ### Metrics To tune metrics, open the card’s menu and select **Advanced metrics tuning**. Metrics run directly on your Cluster. These settings are configured locally and aren’t managed through Grafana Cloud. 1. Switch these signals on or off, then click **Save**. - **Cluster metrics**: Collects CPU, memory, network, and storage metrics from Cluster Nodes and Pods. What is collectedWhy enable? What is collected Why enable? Node resource usage; metrics about Pod health; Persistent volume usage; and Deployment, StatefulSet, and DaemonSet status. Essential for Cluster health monitoring; powers the Kubernetes Overview dashboard; tracks resource utilization and capacity planning; and detects Pod crashes, OOM kills, and so on. - [**Cost metrics**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/manage-costs/) (`costMetrics`): Collects resource costs per namespace and workload to identify expensive Pods and optimize spending. Uses [OpenCost](https://opencost.io/). What is collectedWhy enable?Private registry settings What is collected Why enable? Private registry settings Resource cost attribution by namespace, workload, and Pod; CPU and memory cost breakdowns; Cloud provider pricing data integration; and cost efficiency metrics. Shows cost data; helps identify expensive workloads; enables FinOps and cost optimization; and tracks spending trends over time. To configure OpenCost’s image source in a private registry environment, complete the following steps. Without this setting, OpenCost pods become stuck in ImagePullBackOff and no cost metrics will be collected. 1. Add this to your `values.yaml` file: YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml telemetryServices: opencost: deploy: true opencost: exporter: image: registry: my.private.registry.example.com ``` 2. Then deploy or upgrade: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash helm upgrade --install grafana-k8s-monitoring grafana/k8s-monitoring \ --values values.yaml ``` - [**Energy metrics**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/optimize-resource-usage/monitor-energy-use/) (`hostMetrics.energyMetrics`): Measures power consumption per Node and workload for carbon footprint tracking. Uses [Kepler](https://sustainable-computing.io/). - [**Pod and Service metrics by annotation**](https://github.com/grafana/k8s-monitoring-helm/blob/main/charts/k8s-monitoring/docs/examples/features/annotation-autodiscovery/default/README.md): Scrapes metrics from Pods and Services that have the `k8s.grafana.com/scrape` annotation. What does this do?Why enable?Why disable? What does this do? Why enable? Why disable? Automatically discovers running pods and services; dynamically scrapes Prometheus metrics from annotated Pods; detects new workloads without manual configuration; and uses Kubernetes annotations to find metrics endpoints. Enable for: zero-configuration metrics collection; automatically monitor new applications as they deploy; support microservices architectures with dynamic scaling; and find application-specific metrics beyond system metrics. Disable when: You want to explicitly define every scrape target; you’re concerned about discovering unintended metrics endpoints; your cluster has very strict network policies. - [**Prometheus Operator objects**](https://github.com/grafana/k8s-monitoring-helm/blob/main/charts/k8s-monitoring/docs/examples/features/prometheus-operator-objects/default/README.md): Adds scrape targets from ServiceMonitor, PodMonitor, and Probe objects. What does this do?Why enable?Why disable? What does this do? Why enable? Why disable? Discovers and monitors Prometheus Operator CRDs: ServiceMonitor, which defines how to scrape metrics from Kubernetes services; PodMonitor, which defines how to scrape metrics from Pods; and Probe, which defines blackbox probing of endpoints. Enable when: you’re already using Prometheus Operator in your Cluster; you have existing ServiceMonitor/PodMonitor definitions; you want to leverage existing Prometheus configurations; you want to enable migration from Prometheus Operator to Grafana Alloy. Disable when: You’re not using Prometheus Operator Objects; you prefer using the native configuration of Grafana Alloy. ### Logs and Events To tune logs and events, open the card’s menu and select **Advanced logs and events tuning**. These options are deployed and controlled through Grafana Cloud, so you can update and scale instrumentation without changing your Cluster configuration. 1. Switch these signals on or off, then click **Save**. - [**Cluster events**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/manage-configuration/#cluster-events): Collects Kubernetes events, including Pod failures, scheduling events, and resource state changes. What is collectedWhy enable? What is collected Why enable? Generated from: the scheduler (assigning Pods to Nodes); `kubelet` (managing Pods on Nodes); the controller manager (handling scaling, deployments, and so on); the API server (processing requests). Troubleshooting scheduling & deployment issues; see why a Pod isn’t starting (no Nodes with enough memory); tracking resource lifecycle changes (Pod created → scheduled → pulled → started → ready → terminated); detecting transient or recurring failures (repeated image pull errors, failed probes, or node taints); auditing Cluster activity (to identify which controller or user triggered changes). - [**Node logs**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/manage-configuration/#node-logs): Collects `kubelet` and system logs from worker Nodes running systemd, including [journald](https://www.freedesktop.org/software/systemd/man/latest/systemd-journald.service.html) logs. What is collectedWhy enable? What is collected Why enable? Logs from: the operating system (kernel logs, systemd, network drivers); Kubernetes node agents (like `kubelet`); container runtimes (Docker, containerd); system daemons (journald, syslog). Diagnosing node instability (memory exhaustion, CPU throttling, or disk space issues); debugging scheduling or startup failures (when Pods can’t start, the issue may be at the Node level); investigating network or storage problems; determining driver or volume mount failures; auditing system changes (Node reboots, `kubelet` restarts, or OS updates). - [**Pod logs**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/manage-configuration/#pod-logs): Collects application logs from running Pods. Available as `podLogsViaLoki` (Loki pipeline) or `podLogsViaOpenTelemetry` (native OTLP format) in the Helm chart values. What is collectedWhy enable? What is collected Why enable? Standard output (stdout) and standard error (stderr) Pod logs from the processes in the container, such as initialization messages, API request logs, warnings or errors, and application-specific information. Enable for: debugging issues, such as when an application crashes or behaves unexpectedly, logs reveal what went wrong; monitoring behavior, such as tracking normal operational messages (startup confirmation, API requests, or job completion); auditing events, to view logs that show what actions were taken by your app or scripts running inside containers; gaining performance insight, such as tracing slow operations or bottlenecks using timestamps and log levels. - [**Kubernetes manifests**](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/troubleshooting/#investigate-workload-changes) (`kubernetesManifests`): Collects Kubernetes resource manifests and changes as logs, using [`k8s-manifest-tail`](https://github.com/grafana/k8s-manifest-tail). Powers the **View Manifest** and **Changes** tabs on workloads and Pods. What is collectedWhy enable? What is collected Why enable? Full resource manifests for Pods and workloads; creation, update, and deletion events for those resources; and the version history that powers the manifest diff viewer. When switched, you can compare a manifest against previous versions in a side-by-side diff; on Deployment detail page of `k8s-manifest-tail`, you can view a highlight on the graphs in the Optimization section to see if a deployment change correlates with a change in behavior. ### Application observability To configure application observability, open the card’s menu and select **Advanced settings**. These options change the Helm chart’s collector shape and are emitted into the install command. They aren’t configurable through Fleet Management. 1. Switch these signals on or off, then click **Save**. - **OpenTelemetry receivers**: Enables Alloy to receive metrics, logs, and traces from applications instrumented with OpenTelemetry. What is collectedWhat does this do?Why enable?Why disable? What is collected What does this do? Why enable? Why disable? Metrics, logs, and traces from apps sending OTel data. Opens Alloy receiver ports to accept telemetry that your apps (or OTel SDK/collector) push to Grafana Cloud. Enable for: services using OTel SDKs so you can push traces/metrics to an endpoint; Application Observability (RED metrics, service map, trace correlation) in Grafana Cloud; existing Zipkin-instrumented apps when you want to receive traces from them; ensuring host-hours telemetry, which is required for Application Observability billing. Disable when: your apps have no instrumentation at all. - **Application profiling**: Captures CPU and memory profiles from Pods that have the `profiles.grafana.com/scrape` annotation. What does this do?Why enable?Why disable? What does this do? Why enable? Why disable? Enables continuous profiling using eBPF, Java, or pprof; collects CPU flame graphs from running applications; captures function-level performance data; identifies code issues and performance bottlenecks. Enable when you want to: find expensive functions in your code; optimize application performance; debug CPU-intensive operations; identify memory allocation patterns. Disable when: You don’t need code-level profiling; you’re concerned about profiling overhead (~1-5% CPU); your applications are already well-optimized. - **Zero-code instrumentation**: Captures HTTP, gRPC, and database calls with no code changes, using eBPF and [Grafana Beyla](/docs/beyla/latest/). > Caution > > If you enable instrumentation with Beyla, this may affect your billing due to additional telemetry ingestion. What does this do?Why enable?Why disable? What does this do? Why enable? Why disable? Correlates Pod metrics with application traces; links infrastructure metrics to application performance; enables unified views of resource usage and request patterns; powers the correlation features in Pod detail views. Enable when you want to: See how Pod resource constraints affect application latency; correlate OOM kills with specific requests; understand resource consumption per endpoint; and connect infrastructure issues to user impact. Disable when: You only need basic infrastructure monitoring. - **Forward traces to application receivers**: Forwards traces captured by Beyla to the application receiver OTLP endpoint. Available only when both OpenTelemetry receivers and zero-code instrumentation are enabled. ## Deployment 1. In the **Cluster name** box, enter a unique, human-readable name for this Cluster. Use only lowercase letters, numbers, and hyphens, start with a letter, and end with a letter or number (up to 63 characters). This name appears in dashboards, alerts, and Cluster selectors. 2. In the **Namespace** box, enter the namespace where you want to install Grafana Alloy. 3. Optionally, under **Custom labels**, add key and value labels to help you search for, identify, and deploy pipelines in Fleet Management. Click **Add more** to add another label. 4. If you don’t already have the Grafana Helm charts repository, add it: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash helm repo add grafana https://grafana.github.io/helm-charts ``` If you already have it, update it: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash helm repo update ``` 5. Under **Deployment code to copy**, copy the generated `helm upgrade --install` command and run it in your terminal. Make sure your `kubectl` context points to the Cluster you want to monitor. 6. Click **Test connection** to verify that the collectors install and send data to Grafana Cloud. ### Configure application instrumentation If you chose to forward traces to application receivers, a list of endpoints appear. In your application that generates metrics, logs, or traces, enter the appropriate OTLP or Zipkin address. > Note > > If you change the deployment name to something other than grafana-k8s-monitoring, the endpoint address is updated as well. Be sure to update your applications to point to the correct endpoint. Click **See cluster status** to view the status of data collection. Your data becomes populated as the system components begin scraping and sending data to Grafana Cloud. This view shows the health of the different sources of metrics, Pod logs, and Cluster events, as well as any applicable version numbers. ### Troubleshoot Refer to [Troubleshooting](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/troubleshooting/) for any issues that occur after configuration. ### Install any integrations You can use Grafana integrations to monitor the health and status of services and applications running in your Kubernetes clusters. To install a Kubernetes integration to begin scraping metrics: 1. From the main menu, navigate to **Connections**, and filter for Kubernetes. 2. Select the integration for the service you want to monitor. 3. Follow the instructions on the screen to copy and use the configuration snippet and install the integration. 4. After installing an integration, redeploy the configuration using the method you originally used. ### Retrieve Helm values If you installed Kubernetes Monitoring with the Helm CLI, you can retrieve the values for your configuration by using the [`helm get values` command](https://helm.sh/docs/helm/helm_get_values/).