xk6-disruptor examples on Grafana Labs

Inject gRPC faults into Service

Mon, 21 Sep 2026 15:16:28 +0000

Inject gRPC faults into Service

This example shows a way to use the ServiceDisruptor to test the effect of faults injected in the gRPC requests served by a service.

The complete source code is at the end of this document. The next sections examine the code in detail.

The example uses grpcpbin, a simple request/response application that offers endpoints for testing different gRPC requests.

The test requires grpcpbin to be deployed in a cluster in the namespace grpcbin and exposed with an external IP. The IP address is expected in the environment variable GRPC_HOST.

For the Kubernetes manifests and the instructions on how to deploy it to a cluster, refer to the test setup section at the end of this document. To learn more about how to get the external IP address, refer to Expose your application.

Initialization

The initialization code imports the external dependencies required by the test. The ServiceDisruptor class imported from the xk6-disruptor extension provides functions for injecting faults in services. The k6/net/grpc module provides functions for executing gRPC requests. The check function verifies the results from the requests.

import { ServiceDisruptor } from 'k6/x/disruptor';
import grpc from 'k6/net/grpc';
import { check } from 'k6';

We also create a grpc client and load the protobufs definitions for the HelloService service.

const client = new grpc.Client();
client.load(['pb'], 'hello.proto');

Test Load

The test load is generated by the default function, which connects to the grpcbin service using the IP and port obtained from the environment variable GRPC_HOST and invokes the SayHello method of the hello.HelloService service. Finally, The status code of the response is checked. When faults are injected, this check should fail.

export default function () {
  client.connect(__ENV.GRPC_HOST, {
    plaintext: true,
    timeout: '1s',
  });

  const data = { greeting: 'Bert' };
  const response = client.invoke('hello.HelloService/SayHello', data, {
    timeout: '1s',
  });
  client.close();

  check(response, {
    'status is OK': (r) => r && r.status === grpc.StatusOK,
  });
}

Fault injection

The disrupt function creates a ServiceDisruptor for the grpcbin service in the namespace grpcbin.

The gRPC faults are injected by calling the ServiceDisruptor.injectGrpcFaults method using a fault definition that introduces a delay of 300ms on each request and an error with status code 13 (“Internal error”) in 10% of the requests.

export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }

  const fault = {
    averageDelay: '300ms',
    statusCode: grpc.StatusInternal,
    errorRate: 0.1,
    port: 9000,
  };
  const disruptor = new ServiceDisruptor('grpcbin', 'grpcbin');
  disruptor.injectGrpcFaults(fault, '30s');
}

Notice the following code snippet in the injectFaults function above:

export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }
  //...
}

This code makes the function return without injecting faults if the SKIP_FAULTS environment variable is passed to the execution of the test with a value of “1”. We will use this option to obtain a baseline execution without faults.

Scenarios

This test defines two scenarios to be executed. The load scenario applies the test load to the grpcpbin application for 30s invoking the default function. The disrupt scenario invokes the disrupt function to inject a fault in the gRPC requests to the target application.

export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};

Note
The disrupt scenario uses a shared-iterations executor with one iteration and one VU. This setting ensures the disrupt function is executed only once. Executing this function multiples times concurrently may have unpredictable results.

Executions

Note
The commands in this section assume the xk6-disruptor binary is available in your current directory. This location can change depending on the installation process and the platform. Refer to Installation for details on how to install it in your environment.

Baseline execution

We will first execute the test without introducing faults to have an baseline using the following command:

xk6-disruptor --env SKIP_FAULTS=1 --env GRPC_HOST=$GRPC_HOST run grpc-faults.js

xk6-disruptor --env SKIP_FAULTS=1 --env "GRPC_HOST=$Env:GRPC_HOST" run grpc-faults.js

Notice the argument --env SKIP_FAULT=1, which makes the disrupt function return without injecting any fault as explained in the fault injection section. Also notice the --env GRPC_HOST argument, which passes the external IP used to access the grpcbin application.

You should get an output similar to the following (use the Expand button to see all output).

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: scripts/grpc-faults.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m30.0s), 000/011 VUs, 3001 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m00.0s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/010 VUs  30s             100.00 iters/s

  █ TOTAL RESULTS

    checks_total.......................: 3000    100.03/s
    checks_succeeded...................: 100.00% 3000 out of 3000
    checks_failed......................: 0.00%   0 out of 3000

    ✓ status is OK

    GRPC
    grpc_req_duration..................: avg=1.36ms min=512.43µs med=1.25ms max=26.62ms p(90)=1.75ms p(95)=2.09ms

    EXECUTION
    iteration_duration.................: avg=4.4ms  min=52.86µs  med=4.01ms max=66.26ms p(90)=5.32ms p(95)=6.35ms
    iterations.........................: 3001    100.03/s
    vus................................: 10      min=10       max=10
    vus_max............................: 11      min=11       max=11

    NETWORK
    data_received......................: 416 kB  14 kB/s
    data_sent..........................: 630 kB  21 kB/s

Fault injection

We repeat the execution injecting the faults. Notice we have removed the --env SKIP_FAULTS=1 argument.

xk6-disruptor --env GRPC_HOST=$GRPC_HOST run grpc-faults.js

xk6-disruptor --env "GRPC_HOST=$Env:GRPC_HOST" run grpc-faults.js

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: scripts/grpc-faults.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m50.6s), 000/033 VUs, 2975 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m50.6s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/032 VUs  30s             100.00 iters/s

  █ TOTAL RESULTS

    checks_total.......................: 2974    58.77/s
    checks_succeeded...................: 89.71%  2668 out of 2974
    checks_failed......................: 10.29%  306 out of 2974

    ✗ status is OK
     ↳  89% — ✓ 2668 / ✗ 306

    GRPC
    grpc_req_duration..................: avg=270.51ms min=502.12µs med=302.34ms max=310.48ms p(90)=303.25ms p(95)=303.57ms

    EXECUTION
    dropped_iterations.................: 26     0.51/s
    iteration_duration.................: avg=290.05ms min=1.96ms   med=304.86ms max=50.61s   p(90)=306.36ms p(95)=306.89ms
    iterations.........................: 2975   58.77/s
    vus................................: 1      min=1        max=33
    vus_max............................: 33     min=25       max=33

    NETWORK
    data_received......................: 405 kB 8.0 kB/s
    data_sent..........................: 615 kB 12 kB/s

Comparison

Let’s take a closer look at the results for the requests on each scenario. We can observe that in the base scenario requests duration has an percentile 95 of 2.09ms and 100% of requests pass the check. The faults scenario has a percentile 96 of 303.57ms and only 89% of requests pass the check (or put in another way, 11% of requests failed), closely matching the fault definition.

Execution	P95 Req. Duration	Passed Checks
Baseline	2.09ms	100%
Fault injection	303.57ms	89%

Source Code

import { ServiceDisruptor } from 'k6/x/disruptor';
import grpc from 'k6/net/grpc';
import { check } from 'k6';

const client = new grpc.Client();
client.load(['pb'], 'hello.proto');

export default function () {
  client.connect(__ENV.GRPC_HOST, {
    plaintext: true,
    timeout: '1s',
  });

  const data = { greeting: 'Bert' };
  const response = client.invoke('hello.HelloService/SayHello', data, {
    timeout: '1s',
  });
  client.close();

  check(response, {
    'status is OK': (r) => r && r.status === grpc.StatusOK,
  });
}

export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }

  const disruptor = new ServiceDisruptor('grpcbin', 'grpcbin');

  // inject errors in requests
  const fault = {
    averageDelay: '300ms',
    statusCode: grpc.StatusInternal,
    errorRate: 0.1,
    port: 9000,
  };
  disruptor.injectGrpcFaults(fault, '30s');
}

export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};

syntax = "proto2";

package hello;

service HelloService {
  rpc SayHello(HelloRequest) returns (HelloResponse);
  rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse);
  rpc LotsOfGreetings(stream HelloRequest) returns (HelloResponse);
  rpc BidiHello(stream HelloRequest) returns (stream HelloResponse);
}

message HelloRequest {
  optional string greeting = 1;
}

message HelloResponse {
  required string reply = 1;
}

Test setup

The tests requires the deployment of the grpcbin application. The application must also be accessible using an external IP available in the GRPC_HOST environment variable.

The following manifests define the resources required for deploying the application and exposing it as a LoadBalancer service.

You can deploy the application using the following commands:

# Create Namespace
kubectl apply -f namespace.yaml
namespace/grpcbin created

# Deploy Pod
kubectl apply -f pod.yaml
pod/grpcbin created

# Expose Pod as a Service
kubectl apply -f service.yaml
service/grpcbin created

You must set the environment variable GRPC_HOST with the external IP address and port used to access the grpcbin service from the test script.

Learn more about how to get the external IP address in the Expose your application.

Manifests

kind: 'Pod'
apiVersion: 'v1'
metadata:
  name: grpcbin
  namespace: grpcbin
  labels:
    app: grpcbin
spec:
  containers:
    - name: grpcbin
      image: moul/grpcbin
      ports:
        - name: http
          containerPort: 9000

apiVersion: 'v1'
kind: 'Service'
metadata:
  name: grpcbin
  namespace: grpcbin
spec:
  selector:
    app: grpcbin
  type: 'NodePort'
  ports:
    - port: 9000
      targetPort: 9000

Inject HTTP faults into Pod

Mon, 21 Sep 2026 15:16:28 +0000

Inject HTTP faults into Pod

This example shows how PodDisruptor can be used for testing the effect of faults injected in the HTTP requests served by a pod.

You will find the complete source code at the end of this document. Next sections examine the code in detail.

The example uses httpbin, a simple request/response application that offers endpoints for testing different HTTP requests.

The test requires httpbin to be deployed in a cluster in the namespace httpbin and exposed with an external IP. the IP address is expected in the environment variable SVC_IP.

You will find the Kubernetes manifests and the instructions of how to deploy it to a cluster in the test setup section at the end of this document. You can learn more about how to get the external IP address in the expose your application section.

Initialization

The initialization code imports the external dependencies required by the test. The PodDisruptor class imported from the xk6-disruptor extension provides functions for injecting faults in pods. The k6/http module provides functions for executing HTTP requests.

import { PodDisruptor } from 'k6/x/disruptor';
import http from 'k6/http';

Test Load

The test load is generated by the default function, which executes a request to the httpbin pod using the IP obtained from the environment variable SVC_IP. The test makes requests to the endpoint delay/0.1 which will return after 0.1 seconds (100ms).

export default function (data) {
  http.get(`http://${data.SVC_IP}/delay/0.1`);
}

Note
The test uses the delay endpoint which return after the requested delay. It requests a 0.1s (100ms) delay to ensure the baseline scenario (see scenarios below) has meaningful statistics for the request duration. If we were simply calling a locally deployed http server (for example nginx), the response time would exhibit a large variation between a few microseconds to a few milliseconds. Having 100ms as baseline response time has proved to offer more consistent results.

Fault injection

The disrupt function creates a PodDisruptor using a selector that matches pods in the namespace httpbin with the label app: httpbin.

The http faults are injected by calling the PodDisruptor.injectHTTPFaults method using a fault definition that introduces a delay of 50ms on each request and an error code 500 in 10% of the requests.

export function disrupt(data) {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }
  const selector = {
    namespace: namespace,
    select: {
      labels: {
        app: 'httpbin',
      },
    },
  };
  const podDisruptor = new PodDisruptor(selector);
  // delay traffic from one random replica of the deployment
  const fault = {
    averageDelay: '50ms',
    errorCode: 500,
    errorRate: 0.1,
  };
  podDisruptor.injectHTTPFaults(fault, '30s');
}

Notice the following code snippet in the injectFaults function above:

export function disrupt(data) {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }
  //...
}

Scenarios

This test defines two scenarios to be executed. The load scenario applies the test load to the httpbin application for 30s invoking the default function. The disrupt scenario invokes the disrupt function to inject a fault in the HTTP requests of the target application.

export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};

Note
Notice that the disrupt scenario uses a shared-iterations executor with one iteration and one VU. This setting ensures the disrupt function is executed only once. Executing this function multiples times concurrently may have unpredictable results.

Executions

Note
The commands in this section assume the xk6-disruptor binary is available in your current directory. This location can change depending on the installation process and the platform. Refer to the installation section for details on how to install it in your environment.

Baseline execution

We will first execute the test without introducing faults to have an baseline using the following command:

xk6-disruptor run --env SKIP_FAULTS=1 --env SVC_IP=$SVC_IP disrupt-pod.js

xk6-disruptor run --env SKIP_FAULTS=1 --env "SVC_IP=$Env:SVC_IP" disrupt-pod.js

Notice the argument --env SKIP_FAULT=1, which makes the disrupt function return without injecting any fault as explained in the fault injection section. Also notice the --env SVC_IP argument, which passes the external IP used to access the httpbin application.

You should get an output similar to the one shown below (click Expand button to see all output).

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: test.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m30.1s), 000/014 VUs, 2998 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m00.0s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/013 VUs  30s             100.00 iters/s

     data_received..................: 1.4 MB 46 kB/s
     data_sent......................: 267 kB 8.9 kB/s
     dropped_iterations.............: 4      0.132766/s
     http_req_blocked...............: avg=8.08µs   min=2.36µs   med=5.8µs    max=543.79µs p(90)=8.68µs   p(95)=10.5µs
     http_req_connecting............: avg=1.25µs   min=0s       med=0s       max=418.63µs p(90)=0s       p(95)=0s
     http_req_duration..............: avg=103.22ms min=101.65ms med=103.13ms max=121.7ms  p(90)=104.01ms p(95)=104.4ms
       { expected_response:true }...: avg=103.22ms min=101.65ms med=103.13ms max=121.7ms  p(90)=104.01ms p(95)=104.4ms
     http_req_failed................: 0.00%  ✓ 0         ✗ 2997
     http_req_receiving.............: avg=133.5µs  min=45.06µs  med=131.23µs max=879.43µs p(90)=193.48µs p(95)=223.04µs
     http_req_sending...............: avg=31.14µs  min=11.46µs  med=29.37µs  max=171.68µs p(90)=40.48µs  p(95)=47.53µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=103.05ms min=101.54ms med=102.98ms max=121.56ms p(90)=103.82ms p(95)=104.2ms
     http_reqs......................: 2997   99.474844/s
     iteration_duration.............: avg=103.34ms min=109.86µs med=103.28ms max=121.92ms p(90)=104.19ms p(95)=104.63ms
     iterations.....................: 2998   99.508035/s
     vus............................: 13     min=11      max=13
     vus_max........................: 14     min=12      max=14

Fault injection

We repeat the execution injecting the faults. Notice we have removed the --env SKIP_FAULTS=1 argument.

xk6-disruptor run --env SVC_IP=$SVC_IP disrupt-pod.js

xk6-disruptor run --env "SVC_IP=$Env:SVC_IP" disrupt-pod.js

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: disrupt-pod.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m31.1s), 000/018 VUs, 2995 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m31.1s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/017 VUs  30s             100.00 iters/s

     data_received..................: 1.1 MB 34 kB/s
     data_sent......................: 267 kB 8.6 kB/s
     dropped_iterations.............: 7      0.224798/s
     http_req_blocked...............: avg=9.81µs   min=2.59µs   med=5.93µs   max=489.67µs p(90)=7.88µs   p(95)=9.5µs
     http_req_connecting............: avg=2.48µs   min=0s       med=0s       max=367.63µs p(90)=0s       p(95)=0s
     http_req_duration..............: avg=142.15ms min=50.33ms  med=153.79ms max=165.85ms p(90)=154.8ms  p(95)=155.12ms
       { expected_response:true }...: avg=151.9ms  min=101.81ms med=153.9ms  max=165.85ms p(90)=154.86ms p(95)=155.17ms
     http_req_failed................: 9.65%  ✓ 289       ✗ 2705
     http_req_receiving.............: avg=80.92µs  min=28.32µs  med=77.33µs  max=352.19µs p(90)=105.09µs p(95)=123.68µs
     http_req_sending...............: avg=30.43µs  min=11.27µs  med=29.37µs  max=287.71µs p(90)=37.42µs  p(95)=41.84µs
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s
     http_req_waiting...............: avg=142.04ms min=50.25ms  med=153.68ms max=165.76ms p(90)=154.69ms p(95)=155.01ms
     http_reqs......................: 2994   96.149356/s
     iteration_duration.............: avg=152.64ms min=50.43ms  med=153.93ms max=31.12s   p(90)=154.97ms p(95)=155.29ms
     iterations.....................: 2995   96.18147/s
     vus............................: 1      min=1       max=18
     vus_max........................: 18     min=12      max=18

Comparison

Let’s take a closer look at the results for the requests on each scenario. We can observe that he base scenario has an average of 103ms and an error rate of 0% while the faults scenario has a median around 151.9ms and an error rate of nearly 10%, matching the definition of the faults defined in the disruptor.

Execution	Avg. Response	Failed requests
Baseline	103.22ms	0.00%
Fault injection	151.9 ms	9.65%

Note
Notice we have used the average response time reported as expected_response:true because this metric only consider successful requests while http_req_duration considers all requests, including those returning a fault.

Source Code

import { PodDisruptor } from 'k6/x/disruptor';
import http from 'k6/http';

export default function (data) {
  http.get(`http://${__ENV.SVC_IP}/delay/0.1`);
}

export function disrupt(data) {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }

  const selector = {
    namespace: 'httpbin',
    select: {
      labels: {
        app: 'httpbin',
      },
    },
  };
  const podDisruptor = new PodDisruptor(selector);

  // delay traffic from one random replica of the deployment
  const fault = {
    averageDelay: '50ms',
    errorCode: 500,
    errorRate: 0.1,
  };
  podDisruptor.injectHTTPFaults(fault, '30s');
}

export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};

Test setup

The tests requires the deployment of the httpbin application. The application must also be accessible using an external IP available in the SVC_IP environment variable.

The manifests below define the resources required for deploying the application and exposing it as a LoadBalancer service.

You can deploy the application using the following commands:

# Create Namespace
kubectl apply -f namespace.yaml
namespace/httpbin created

# Deploy Pod
kubectl apply -f pod.yaml
pod/httpbin created

# Expose Pod as a Service
kubectl apply -f service.yaml
service/httpbin created

You can retrieve the resources using the following command:

kubectl -n httpbin get all
NAME          READY   STATUS    RESTARTS   AGE
pod/httpbin   1/1     Running   0          1m

NAME              TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
service/httpbin   LoadBalancer   10.96.169.78   172.18.255.200   80:31224/TCP   1m

You must set the environment variable SVC_IP with the external IP address and port used to access the httpbin service from the test script.

You can learn more about how to get the external IP address in the expose your application section.

Manifests

kind: 'Pod'
apiVersion: 'v1'
metadata:
  name: httpbin
  namespace: httpbin
  labels:
    app: httpbin
spec:
  containers:
    - name: httpbin
      image: kennethreitz/httpbin
      ports:
        - name: http
          containerPort: 80

apiVersion: 'v1'
kind: 'Service'
metadata:
  name: httpbin
  namespace: httpbin
spec:
  selector:
    app: httpbin
  type: 'LoadBalancer'
  ports:
    - port: 80
      targetPort: 80

Interactive demo

Mon, 21 Sep 2026 15:16:28 +0000

Interactive demo

Visit the interactive demo environment in Killercoda and try the disruptor in a demo application without having to do any setup.