Faults on Grafana Labs

gRPC

Mon, 21 Sep 2026 15:16:28 +0000

gRPC

A gRPC Fault describes the characteristics of the faults to be injected in the gRPC requests served by a target.

A gRPC fault is described by the following attributes:

Attribute	Type	Description
averageDelay	string	average delay added to requests represented as a string (default `0`)
delayVariation	string	variation in the injected delay (default `0`)
statusMessage	string	message to be returned when an error is injected
statusCode	number	status to be returned when an error is injected
errorRate	number	rate of requests that will return an error, represented as a float in the range `0.0` to `1.0` (default `0.0`)
exclude	string	comma-separated list of services to be excluded from disruption
port	number	port on which the requests will be intercepted

Note
averageDelay and delayVariation are applied to all requests affected by the fault, regardless of the value of errorRate. statusCode is returned only to a fraction of requests defined by errorRate.

Example

This example defines a gRPC fault that introduces a delay of 50ms in all requests and returns a status code 13 in 10% of the requests.

const fault = {
  averageDelay: '50ms',
  statusCode: 10,
  errorRate: 0.1,
};

HTTP

Mon, 21 Sep 2026 15:16:28 +0000

HTTP

A HTTP Fault describes the characteristics of the faults to be injected in the HTTP requests served by a target.

A HTTP fault is described by the following attributes:

Attribute	Type	Description
averageDelay	string	average delay added to requests represented as a string (default `0`)
delayVariation	string	variation in the injected delay (default `0`)
errorBody	string	body to be returned when an error is injected
errorCode	number	error code to return
errorRate	number	rate of requests that will return an error, represented as a float in the range `0.0` to `1.0` (default `0.0`)
exclude	string	comma-separated list of urls to be excluded from disruption (e.g. /health)
port	number	port on which the requests will be intercepted

Note
averageDelay and delayVariation are applied to all requests affected by the fault, regardless of the value of errorRate. errorCode is returned only to a fraction of requests defined by errorRate.

Example

This example defines a HTTP fault that introduces a delay of 50ms in all requests and returns an error code 500 in 10% of the requests.

const fault = {
  averageDelay: '50ms',
  errorCde: 500,
  errorRate: 0.1,
};

Pod Termination

Mon, 21 Sep 2026 15:16:28 +0000

Pod Termination

A Pod Termination Fault allows terminating either a fixed number or a percentage of the pods that matching a selector or back a service.

A Pod Termination fault is defined by the following attributes:

Attribute	Type	Description
count	integer or percentage	the number of pods to be terminated. It can be specified as a integer number or as a percentage, for example `30%`, that defines the fraction of target pods to be terminated

Note
If the count is a percentage and there are no enough elements in the target pod list, the number is rounded up. For example ‘25%’ of a list of 2 target pods will terminate one pod. If the list of target pods is not empty, at least one pod is always terminated.

Example

This example defines a PorTermination fault that will terminate 30% of target pods

const fault = {
  count: '30%',
};