Grafana Cloud Performance testing Author and run tests Set up private load zones

Set up private load zones

In Grafana Cloud k6, there are 20+ load zones that you can use to run your tests from different locations. But what happens if you:

• Want to run your tests from a location that’s not included in the default load zones?
• Want to test an internal service that isn’t exposed to the internet?

Private Load Zones (PLZ) are load zones that you can host inside your network. They can run on top of any Kubernetes cluster you have. These PLZs are built on top of the existing k6-operator project as an additional custom resource definition (CRD). You can start a cloud test in a PLZ by referencing it by name from your script, and the test will run in the nodes of your Kubernetes cluster.

Let’s see how to set up a Private Load Zone and run a test.

Before you begin

To set up and use a Private Load Zone, you’ll need:

1. A Grafana Cloud account.
2. A Kubernetes cluster (version 1.23 or higher):
   • That can access the APIs and services you wish to test.
   • That can access “https://ingest.k6.io” and “https://api.k6.io”

Set up a Private Load Zone

First, you need to set up a Private Load Zone in your Kubernetes cluster.

Install k6-operator

There are two ways to install k6-operator in your cluster.

With bundle

Install k6-operator by using the k6-operator YAML bundle generated on every release:

bash Copy

kubectl apply -f https://raw.githubusercontent.com/grafana/k6-operator/main/bundle.yaml

By default, k6-operator is installed into the k6-operator-system namespace. You can check that it’s running there with:

bash Copy

kubectl get pods -n k6-operator-system

And the output should look similar to:

bash Copy

NAME                                              READY   STATUS    RESTARTS   AGE
k6-operator-controller-manager-6f88ddf8df-7vvgx   2/2     Running   0          12s

With Helm

Install k6-operator by using the Helm chart:

bash Copy

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install k6-operator grafana/k6-operator

With the command above, the operator will be installed into the k6-operator-system namespace. You can check that it’s running there with:

bash Copy

kubectl get pods -n k6-operator-system

And the output should look similar to:

bash Copy

NAME                                              READY   STATUS    RESTARTS   AGE
k6-operator-controller-manager-6f88ddf8df-7vvgx   2/2     Running   0          12s

Create the PLZ CRD

1. Next, you must pick the namespace where you will create your Private Load Zone and where k6 PLZ tests will be executed by creating a custom resource definition (CRD). Namespace can be different from where the k6-operator executes.

   Create a new namespace (replace plz-ns with the name you would like to use):

   bash Copy

   kubectl create namespace plz-ns

2. Generate a new Grafana Stack API Token:

   • Log in to your Grafana Cloud account.
   • Go to the Performance testing app.
   • Select Settings.
   • Select the Grafana Stack API token tab.
   • Select Create token.
   • Give your token a name and select Create new token.
   • Copy and save the token.

   Note: This must be a Grafana Stack API Token and not a personal token.

3. Then, create a Kubernetes secret with the token by using the following command:

   bash Copy

   kubectl create secret generic grafana-k6-token -n plz-ns \
     --from-literal=token=GRAFANA_STACK_API_TOKEN

   Note: You can also use external-secrets or other tools to create a secret.

4. Now, create a definition of your Private Load Zone as a plz.yaml file. Replace PLZ_NAME with the name you’d like to use in your tests:

   bash Copy

   apiVersion: k6.io/v1alpha1
   kind: PrivateLoadZone
   metadata:
     name: PLZ_NAME
     namespace: plz-ns
   spec:
     token: grafana-k6-token
     resources:
       limits:
         cpu: 256m
         memory: 1024Mi

   The token and resources.limits parameters are required. Resources must indicate how much CPU and memory a pod executing a k6 test can take up on a node.

   Note that currently, resources can only be configured during the creation of a Private Load Zone. You’ll have to delete and create a new Private Load Zone if you’d like to update your configuration.

5. Now that the Private Load Zone is defined, create it in your cluster:

   bash Copy

   kubectl apply -f plz.yaml

   Warning: The k6-operator only supports one Private Load Zone per cluster. If you add multiple Private Load Zones to a cluster, your tests might not work correctly.

6. Check that your Private Load Zone was created:

   bash Copy

   kubectl -n plz-ns get privateloadzones.k6.io

Run a test in the PLZ

Once you create a Private Load Zone, any user in your organization with the ability to run a test in Grafana Cloud k6 can run a test using your PLZ.

From the CLI

Make sure that you have authenticated with the k6 CLI.

Then, go to a project or create a new project, and copy the ProjectID by selecting the Copy to clipboard icon.

Create a new file and name it plz-test.js. You can copy the example script below, and make sure to change:

• PLZ_NAME to the Private Load Zone name you defined in the metadata property of your CRD file.
• PROJECT_ID to the projectID you copied in the previous step

js Copy

import http from 'k6/http';
import { check } from 'k6';

export let options = {
  stages: [
    { target: 200, duration: '3m30s' },
    { target: 0, duration: '30s' },
  ],
  ext: {
    loadimpact: {
      projectID: PROJECT_ID,
      distribution: {
        private: { loadZone: 'PLZ_NAME', percent: 100 },
      },
    },
  },
};

export default function () {
  const result = http.get('https://test-api.k6.io/public/crocodiles/');
  check(result, {
    'http response status code is 200': result.status === 200,
  });
}

Now run your test with:

bash Copy

k6 cloud plz-test.js

From the UI

Go to a project, or create a new one.

In the top right corner of the project page, click on Create new test.

You can use the Test Builder or Script Editor to create a new test.

Script Editor

By default, the Script Editor will create an example test script. Replace the load zone section from:

js Copy

'amazon:us:ashburn': { loadZone: 'amazon:us:ashburn', percent: 100 },

To:

js Copy

'private': { loadZone: PLZ_NAME, percent: 100 },

Make sure to replace PLZ_NAME with the Private Load Zone name you defined in the metadata property of your CRD file.

Make other changes to the script as necessary, and select Create and Run to run the test.

Test Builder

By default, your test will be empty. Under Options, select Load zones, and then select the load zone drop-down and pick your PLZ. Add all the requests that your test needs, and select Create and Run to run your test.

Additional information

PLZ configuration options

There are a few additional options you can use in your Private Load Zone definition:

Example of an extended definition:

yaml Copy

apiVersion: k6.io/v1alpha1
kind: PrivateLoadZone
metadata:
  name: PLZ_NAME
  namespace: plz-ns
spec:
  token: grafana-k6-token
  resources:
    limits:
      cpu: 256m
      memory: 1024Mi
    requests:
      cpu: 256m
      memory: 512Mi
  serviceAccountName: plz-sa
  nodeSelector:
    foo: bar

When a test is created, serviceAccountName and nodeSelector are passed to the spec of all pods as spec.serviceAccountName and spec.nodeSelector respectively. Resources are passed only to runner pods.

Test lifecycle

When you run a test in a PLZ, during its creation Grafana Cloud k6 estimates how many pods are required to run such a test. Then, k6-operator will start the test by creating a TestRun CRD with all the parameters GCk6 provides.

If your Kubernetes setup doesn’t have enough resources to run the test, GCk6 will wait for 10 minutes (to wait for auto-scaling or some other event that frees / creates resources). If enough resources are still not available after that time, the test will fail to be scheduled.

After the test is finished, k6-operator will delete the TestRun CRD.