--- title: "Configure remote_write with Helm and kube-prometheus-stack | Grafana Cloud documentation" description: "How to configure remote_write with Helm and kube-prometheus-stack to scrape and send metrics to Grafana Cloud" --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Configure remote\_write with Helm and kube-prometheus-stack You can configure Prometheus’s `remote_write` feature to send cluster metrics to Grafana Cloud using Helm and kube-prometheus-stack. The kube-prometheus-stack Helm chart installs the [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) stack. The kube-prometheus stack configures Prometheus Operator with a default Prometheus-Alertmanager-Grafana stack, and sets up preconfigured Alertmanager alerts. It also configures a set of Prometheus scrape targets, and sets up node-exporter and kube-state-metrics. Prometheus Operator is a sub-component of the kube-prometheus stack. [Prometheus Operator](https://github.com/prometheus-operator/prometheus-operator) implements the Kubernetes Operator pattern for managing a Prometheus-based Kubernetes monitoring stack. A Kubernetes Operator consists of Kubernetes custom resources and controller code that abstract away the management and implementation details of running a given service on Kubernetes. For more about Kubernetes Operators, refer to [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) from the Kubernetes docs. The Prometheus Operator provides a set of Kubernetes Custom Resources that simplify Prometheus, Grafana and Alertmanager deployment and configuration. For example, using the `ServiceMonitor` custom resource, you can configure how to monitor groups of Kubernetes services in YAML manifests. The Operator controller then communicates with the Kubernetes API server to monitor service endpoints and automatically generate the required Prometheus scrape configurations for the configured services. For more about Prometheus Operator, refer to the Prometheus Operator [GitHub repository](https://github.com/prometheus-operator/prometheus-operator). ## Before you begin You must have installed [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) in your Kubernetes cluster using the [Helm package manager](https://helm.sh/). - To install Helm on your local machine, refer to [Install Helm](https://helm.sh/docs/intro/quickstart/#install-helm) - To install kube-prometheus-stack, refer to [Install Chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#install-chart). If you did *not* use Helm to install kube-prometheus, refer to [Configure remote\_write with Prometheus Operator](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/configuration/config-other-methods/prometheus/remote-write-operator/). ## Create a Kubernetes Secret to store Grafana Cloud credentials Create a Kubernetes Secret to store your Grafana Cloud Metrics username and password. 1. To find your username, navigate to your stack in the Cloud Portal, and click **Details** next to the Prometheus panel. Your password corresponds to a Cloud Access Policy token that you can generate by clicking on **Generate now** in this same panel. To create a Cloud Access Policy, refer to [Create a Grafana Cloud Access Policy](/docs/grafana-cloud/security-and-account-management/authentication-and-permissions/access-policies/create-access-policies/). 2. Note your Cloud Prometheus username and password. 3. Run the following command to create a Secret, as in this example `kubepromsecret`: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none kubectl create secret generic kubepromsecret \ --from-literal=username=\ --from-literal=password=''\ -n default ``` > **Note**: If you deployed your monitoring stack in a namespace other than `default`, change the `-n default` flag to the appropriate namespace in the above command. For more about this command, refer to [Managing Secrets using kubectl](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/). ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none kubectl create secret generic kubepromsecret \ --from-literal=username=\ --from-literal=password=''\ -n default ``` Instead of creating the secret directly as in this example, alternatively you can use a manifest file. For more about Kubernetes Secrets, refer to [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) from the Kubernetes docs. ## Create a Helm values file with Prometheus remote\_write configuration Create a Helm values file to define parameters for Prometheus’s remote\_write configuration. A Helm values file allows you to set configuration variables that are passed in to Helm’s object templates. To see the default values file for kube-prometheus-stack, refer to [values.yaml](https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/values.yaml) from the kube-prometheus-stack GitHub repository. Create a `values.yaml` file defining Prometheus’s remote\_write configuration, and then apply the new configuration to kube-prometheus-stack. 1. Open a file named `new_values.yaml` in an editor and paste in the following values: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none prometheus: prometheusSpec: remoteWrite: - url: "" basicAuth: username: name: kubepromsecret key: username password: name: kubepromsecret key: password ``` Set the remote\_write URL and basic\_auth username and password using the Secret created in the previous section. 2. Save and close the file. 3. Apply the changes using the following command: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none helm upgrade -f new_values.yaml [your_release_name] prometheus-community/kube-prometheus-stack ``` Replace `[your_release_name]` with the name of the release you used to install kube-prometheus-stack. You can get a list of installed releases using `helm list`. 4. After running `helm upgrade`, you should see the following output: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none Release "your_release_name" has been upgraded. Happy Helming! NAME: your_release_name LAST DEPLOYED: Mon Dec 7 17:29:03 2020 NAMESPACE: default STATUS: deployed REVISION: 2 NOTES: kube-prometheus-stack has been installed. Check its status by running: kubectl --namespace default get pods -l "release=your_release_name" Visit https://github.com/prometheus-operator/kube-prometheus for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator. ``` ## Verify your updates Verify that your changes have propagated to your running Prometheus instances. 1. Find the name of the Prometheus service. This will be in the format <your\_release\_name>-kube-prometheus-stack-prometheus, but could be truncated to fit naming length restrictions. ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none kubectl --namespace default get service ``` 2. Use port-forward with that Prometheus service by running the following command: ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```none kubectl --namespace default port-forward svc/ 9090 ``` 3. Navigate to `http://localhost:9090` in your browser, and then **Status** and **Configuration**. 4. Verify that the `remote_write` block you appended above has propagated to your running Prometheus instances. 5. Navigate to [Kubernetes Monitoring](/docs/grafana-cloud/monitor-infrastructure/kubernetes-monitoring/navigate-k8s-monitoring/#navigate-to-kubernetes-monitoring), and click **Configuration** on the main menu. 6. Click the **Metrics status** tab to view the data status. Your data begins populating in the view as the system components begin scraping and sending data to Grafana Cloud. [**Metrics status** tab with status indicators for one Cluster](/media/docs/grafana-cloud/k8s/metrics-status-9-18.png)