Steadybit logoResilience Hub

Kafka

ExtensionExtension
New
A Steadybit extension with various actions and check about Kafka.
Install now

Kafka

A Steadybit extension with various actions and check about Kafka.
ExtensionExtension
Install now

Kafka

ExtensionExtension
New
A Steadybit extension with various actions and check about Kafka.
Install now

Kafka

A Steadybit extension with various actions and check about Kafka.
ExtensionExtension
Install now
Go back to list
YouTube content is not loaded by default for privacy reasons.

Introduction to the Kafka Extension

The Steadybit Kafka Extension allows you to create chaos and simultaneously check the behavior of your consumer groups.

Integration and Functionality

Integration of Kafka into Steadybit works via the franz-go client library. The extension-kafka utilizes these capabilities and may require elevated rights on the Kafka side to:

  • List brokers, topics, consumer groups, and offsets
  • Elect leaders for partitions
  • Alter broker configurations
  • Create and delete ACLs
  • Delete records

Integration of Kafka in Steadybit

Alter Broker configuration in various ways:

  • Reduce the max batch size of messages
  • Limit the IO/Network threads per broker
  • Limit the rate of creation of new connections

You can also run these actions:

  • Elect new leaders for partitions
  • Produce records either with a fixed amount or with a fixed rate per second
  • Deny access to a topic for a consumer
  • Trigger a delete record request

In parallel, you can check:

  • The current lag of a topic for a consumer group
  • The state of a consumer group
  • The partition activity for a given topic (leader election, replicas going offline or in-sync, new replica order)

Installation and Setup

To integrate the Kafka extension with your environment, follow our setup guide.

Useful Templates

See all
Validate consumer's behavior when new leader is elected

Verify that your application handles a change of the leader properly.

Motivation

By testing your system's resilience to Kafka leader changes in a partition, you can identify potential weaknesses and take appropriate measures to improve its performance.

Structure

To conduct this experiment, we will ensure that all Kafka topics and producers are ready and that the consumer receives and processes messages correctly. We will then elect a new leader for one partition in Kafka and expect the system to work fine.

Message Queue
Kafka
Recoverability
Kafka consumers
Kafka topics
Check Kafka consumer's reaction to record loss
Intent

Intentionally deny access to the topic for consumers and during this time where consumption is stopped, delete records.

We can check the logs of the consumers to see how they handle the loss of records and also authorization access issues.

Message Queue
Kafka
Recoverability
Kafka consumers
Kafka topics
Start Using Steadybit Today

Get started with Steadybit, and you’ll get access to all of our features to discover the full power of Steadybit. Available for SaaS and on-prem!

Are you unsure where to begin?

No worries, our reliability experts are here to help: book a demo with them!

Cookies simplify provisioning of our services. By using our services, you agree that we use cookies. Further information.