Menu
Open source

Apache Kafka

Apache Kafka is a stream-processing platform for handling real-time data. Using xk6-output-kafka extension, you can send k6 metrics in real-time to Kafka, and, optionally, ingest them from InfluxDB.

Build the k6 version

To build a k6 binary with the extension, first, make sure you have Go and Git installed on your machine.

Then, open your terminal and run the following commands:

bash
# Install xk6
go install go.k6.io/xk6/cmd/xk6@latest

# Build the k6 binary
xk6 build --with github.com/grafana/xk6-output-kafka

... [INFO] Build environment ready
... [INFO] Building k6
... [INFO] Build complete: ./k6

xk6 will create the new k6 binary in the local folder.

Note

To learn more about how to build custom k6 versions, check out xk6.

Run the k6 test

You can configure the broker (or multiple ones), topic and message format directly from the command line parameter like this:

bash
$ k6 run --out xk6-kafka=brokers=broker_host:8000,topic=k6

or if you want multiple brokers:

bash
--out xk6-kafka=brokers={broker1,broker2},topic=k6,format=json

You can also specify the message format k6 will use. By default, it will be the same as the JSON output, but you can also use the InfluxDB line protocol for direct “consumption” by InfluxDB:

bash
--out xk6-kafka=brokers=someBroker,topic=someTopic,format=influxdb

You can even modify some of the format settings such as tagsAsFields:

bash
--out xk6-kafka=brokers=someBroker,topic=someTopic,format=influxdb,influxdb.tagsAsFields={url,myCustomTag}

Options

Here is the full list of options that can be configured and passed to the extension:

NameTypeDescription
K6_KAFKA_BROKERSstringList of brokers
K6_KAFKA_TOPICstringThe name of the topic to be sent
K6_KAFKA_AUTH_MECHANISMstringAuthentication mechanism. Default none.
K6_KAFKA_SASL_USERstringKafka User
K6_KAFKA_SASL_PASSWORDstringKafka User Password
K6_KAFKA_SSLboolean
K6_KAFKA_VERSIONstringKafka version. Default the latest
K6_KAFKA_INSECURE_SKIP_TLS_VERIFYbooleanWhether should ignore TLS verifications
K6_KAFKA_PUSH_INTERVALstringInterval of the metrics’ aggregation and upload to the endpoint
K6_KAFKA_FORMATstringMessage format. json or influxdb
K6_KAFKA_LOG_ERRORbooleanBoolean indicating to log kafka errors

Read more