---
title: "Inject gRPC faults into Service | Grafana k6 documentation"
description: "This example shows how to test the effect of faults injected in the gRPC requests served by a service."
---

> For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt).

# Inject gRPC faults into Service

This example shows a way to use the [ServiceDisruptor](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/xk6-disruptor/servicedisruptor/) to test the effect of faults injected in the gRPC requests served by a service.

The complete [source code](#source-code) is at the end of this document. The next sections examine the code in detail.

The example uses [grpcpbin](https://github.com/moul/grpcbin), a simple request/response application that offers endpoints for testing different gRPC requests.

The test requires `grpcpbin` to be deployed in a cluster in the namespace `grpcbin` and exposed with an external IP. The IP address is expected in the environment variable `GRPC_HOST`.

For the Kubernetes manifests and the instructions on how to deploy it to a cluster, refer to the [test setup](#test-setup) section at the end of this document. To learn more about how to get the external IP address, refer to [Expose your application](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/expose--your-application/).

## Initialization

The initialization code imports the external dependencies required by the test. The [ServiceDisruptor](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/xk6-disruptor/servicedisruptor/) class imported from the `xk6-disruptor` extension provides functions for injecting faults in services. The [k6/net/grpc](/docs/k6/latest/javascript-api/k6-net-grpc/) module provides functions for executing gRPC requests. The [check](/docs/k6/latest/using-k6/checks/) function verifies the results from the requests.

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
import { ServiceDisruptor } from 'k6/x/disruptor';
import grpc from 'k6/net/grpc';
import { check } from 'k6';
```

We also create a grpc client and load the protobufs definitions for the [HelloService](https://github.com/moul/pb/blob/master/hello/hello.proto) service.

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
const client = new grpc.Client();
client.load(['pb'], 'hello.proto');
```

## Test Load

The test load is generated by the `default` function, which connects to the `grpcbin` service using the IP and port obtained from the environment variable `GRPC_HOST` and invokes the `SayHello` method of the `hello.HelloService` service. Finally, The status code of the response is checked. When faults are injected, this check should fail.

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
export default function () {
  client.connect(__ENV.GRPC_HOST, {
    plaintext: true,
    timeout: '1s',
  });

  const data = { greeting: 'Bert' };
  const response = client.invoke('hello.HelloService/SayHello', data, {
    timeout: '1s',
  });
  client.close();

  check(response, {
    'status is OK': (r) => r && r.status === grpc.StatusOK,
  });
}
```

## Fault injection

The `disrupt` function creates a `ServiceDisruptor` for the `grpcbin` service in the namespace `grpcbin`.

The gRPC faults are injected by calling the [ServiceDisruptor.injectGrpcFaults](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/xk6-disruptor/servicedisruptor/injectgrpcfaults/) method using a fault definition that introduces a delay of `300ms` on each request and an error with status code `13` (“Internal error”) in `10%` of the requests.

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }

  const fault = {
    averageDelay: '300ms',
    statusCode: grpc.StatusInternal,
    errorRate: 0.1,
    port: 9000,
  };
  const disruptor = new ServiceDisruptor('grpcbin', 'grpcbin');
  disruptor.injectGrpcFaults(fault, '30s');
}
```

Notice the following code snippet in the `injectFaults` function above:

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }
  //...
}
```

This code makes the function return without injecting faults if the `SKIP_FAULTS` environment variable is passed to the execution of the test with a value of “1”. We will use this option to obtain a [baseline execution](#baseline-execution) without faults.

## Scenarios

This test defines two [scenarios](/docs/k6/latest/using-k6/scenarios/) to be executed. The `load` scenario applies the test load to the `grpcpbin` application for `30s` invoking the `default` function. The `disrupt` scenario invokes the `disrupt` function to inject a fault in the gRPC requests to the target application.

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};
```

> Note
> 
> The `disrupt` scenario uses a `shared-iterations` executor with one iteration and one `VU`. This setting ensures the `disrupt` function is executed only once. Executing this function multiples times concurrently may have unpredictable results.

## Executions

> Note
> 
> The commands in this section assume the `xk6-disruptor` binary is available in your current directory. This location can change depending on the installation process and the platform. Refer to [Installation](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/installation/) for details on how to install it in your environment.

### Baseline execution

We will first execute the test without introducing faults to have an baseline using the following command:

Bash windows-powershell

![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```bash
xk6-disruptor --env SKIP_FAULTS=1 --env GRPC_HOST=$GRPC_HOST run grpc-faults.js
```

windows-powershell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```windows-powershell
xk6-disruptor --env SKIP_FAULTS=1 --env "GRPC_HOST=$Env:GRPC_HOST" run grpc-faults.js
```

Notice the argument `--env SKIP_FAULT=1`, which makes the `disrupt` function return without injecting any fault as explained in the [fault injection](#fault-injection) section. Also notice the `--env GRPC_HOST` argument, which passes the external IP used to access the `grpcbin` application.

You should get an output similar to the following (use the `Expand` button to see all output).

![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```none
         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: scripts/grpc-faults.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m30.0s), 000/011 VUs, 3001 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m00.0s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/010 VUs  30s             100.00 iters/s

  █ TOTAL RESULTS

    checks_total.......................: 3000    100.03/s
    checks_succeeded...................: 100.00% 3000 out of 3000
    checks_failed......................: 0.00%   0 out of 3000

    ✓ status is OK

    GRPC
    grpc_req_duration..................: avg=1.36ms min=512.43µs med=1.25ms max=26.62ms p(90)=1.75ms p(95)=2.09ms

    EXECUTION
    iteration_duration.................: avg=4.4ms  min=52.86µs  med=4.01ms max=66.26ms p(90)=5.32ms p(95)=6.35ms
    iterations.........................: 3001    100.03/s
    vus................................: 10      min=10       max=10
    vus_max............................: 11      min=11       max=11

    NETWORK
    data_received......................: 416 kB  14 kB/s
    data_sent..........................: 630 kB  21 kB/s
```

### Fault injection

We repeat the execution injecting the faults. Notice we have removed the `--env SKIP_FAULTS=1` argument.

Bash windows-powershell

![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```bash
xk6-disruptor --env GRPC_HOST=$GRPC_HOST run grpc-faults.js
```

windows-powershell ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```windows-powershell
xk6-disruptor --env "GRPC_HOST=$Env:GRPC_HOST" run grpc-faults.js
```

![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```none
         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

  execution: local
     script: scripts/grpc-faults.js
     output: -

  scenarios: (100.00%) 2 scenarios, 101 max VUs, 10m30s max duration (incl. graceful stop):
           * disrupt: 1 iterations shared among 1 VUs (maxDuration: 10m0s, exec: disrupt, gracefulStop: 30s)
           * load: 100.00 iterations/s for 30s (maxVUs: 10-100, exec: default, gracefulStop: 30s)


running (00m50.6s), 000/033 VUs, 2975 complete and 0 interrupted iterations
disrupt ✓ [======================================] 1 VUs        00m50.6s/10m0s  1/1 shared iters
load    ✓ [======================================] 000/032 VUs  30s             100.00 iters/s

  █ TOTAL RESULTS

    checks_total.......................: 2974    58.77/s
    checks_succeeded...................: 89.71%  2668 out of 2974
    checks_failed......................: 10.29%  306 out of 2974

    ✗ status is OK
     ↳  89% — ✓ 2668 / ✗ 306

    GRPC
    grpc_req_duration..................: avg=270.51ms min=502.12µs med=302.34ms max=310.48ms p(90)=303.25ms p(95)=303.57ms

    EXECUTION
    dropped_iterations.................: 26     0.51/s
    iteration_duration.................: avg=290.05ms min=1.96ms   med=304.86ms max=50.61s   p(90)=306.36ms p(95)=306.89ms
    iterations.........................: 2975   58.77/s
    vus................................: 1      min=1        max=33
    vus_max............................: 33     min=25       max=33

    NETWORK
    data_received......................: 405 kB 8.0 kB/s
    data_sent..........................: 615 kB 12 kB/s
```

### Comparison

Let’s take a closer look at the results for the requests on each scenario. We can observe that in the `base` scenario requests duration has an percentile 95 of `2.09ms` and `100%` of requests pass the check. The `faults` scenario has a percentile 96 of `303.57ms` and only `89%` of requests pass the check (or put in another way, `11%` of requests failed), closely matching the fault definition.

Expand table

| Execution       | P95 Req. Duration | Passed Checks |
|-----------------|-------------------|---------------|
| Baseline        | 2.09ms            | 100%          |
| Fault injection | 303.57ms          | 89%           |

## Source Code

grpc-faults.js

JavaScript ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```javascript
import { ServiceDisruptor } from 'k6/x/disruptor';
import grpc from 'k6/net/grpc';
import { check } from 'k6';

const client = new grpc.Client();
client.load(['pb'], 'hello.proto');

export default function () {
  client.connect(__ENV.GRPC_HOST, {
    plaintext: true,
    timeout: '1s',
  });

  const data = { greeting: 'Bert' };
  const response = client.invoke('hello.HelloService/SayHello', data, {
    timeout: '1s',
  });
  client.close();

  check(response, {
    'status is OK': (r) => r && r.status === grpc.StatusOK,
  });
}

export function disrupt() {
  if (__ENV.SKIP_FAULTS == '1') {
    return;
  }

  const disruptor = new ServiceDisruptor('grpcbin', 'grpcbin');

  // inject errors in requests
  const fault = {
    averageDelay: '300ms',
    statusCode: grpc.StatusInternal,
    errorRate: 0.1,
    port: 9000,
  };
  disruptor.injectGrpcFaults(fault, '30s');
}

export const options = {
  scenarios: {
    load: {
      executor: 'constant-arrival-rate',
      rate: 100,
      preAllocatedVUs: 10,
      maxVUs: 100,
      exec: 'default',
      startTime: '0s',
      duration: '30s',
    },
    disrupt: {
      executor: 'shared-iterations',
      iterations: 1,
      vus: 1,
      exec: 'disrupt',
      startTime: '0s',
    },
  },
};
```

pb/hello.proto

![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```none
syntax = "proto2";

package hello;

service HelloService {
  rpc SayHello(HelloRequest) returns (HelloResponse);
  rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse);
  rpc LotsOfGreetings(stream HelloRequest) returns (HelloResponse);
  rpc BidiHello(stream HelloRequest) returns (stream HelloResponse);
}

message HelloRequest {
  optional string greeting = 1;
}

message HelloResponse {
  required string reply = 1;
}
```

## Test setup

The tests requires the deployment of the `grpcbin` application. The application must also be accessible using an external IP available in the `GRPC_HOST` environment variable.

The following [manifests](#manifests) define the resources required for deploying the application and exposing it as a LoadBalancer service.

You can deploy the application using the following commands:

Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```bash
# Create Namespace
kubectl apply -f namespace.yaml
namespace/grpcbin created

# Deploy Pod
kubectl apply -f pod.yaml
pod/grpcbin created

# Expose Pod as a Service
kubectl apply -f service.yaml
service/grpcbin created
```

You must set the environment variable `GRPC_HOST` with the external IP address and port used to access the `grpcbin` service from the test script.

Learn more about how to get the external IP address in the [Expose your application](/docs/k6/latest/testing-guides/injecting-faults-with-xk6-disruptor/expose--your-application/).

### Manifests

namespace.yaml

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
apiVersion: 'v1'
kind: Namespace
metadata:
  name: grpcbin
```

pod.yaml

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
kind: 'Pod'
apiVersion: 'v1'
metadata:
  name: grpcbin
  namespace: grpcbin
  labels:
    app: grpcbin
spec:
  containers:
    - name: grpcbin
      image: moul/grpcbin
      ports:
        - name: http
          containerPort: 9000
```

service.yaml

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
apiVersion: 'v1'
kind: 'Service'
metadata:
  name: grpcbin
  namespace: grpcbin
spec:
  selector:
    app: grpcbin
  type: 'NodePort'
  ports:
    - port: 9000
      targetPort: 9000
```