Intent

Verify that an increased latency in your Kafka message delivery is handled by your application properly by having increased processing time but still being able to maintain the throughput.

Motivation

Latency in Kafka can occur due to various reasons, such as network congestion, increased load, or insufficient resources. Such latency can impact the performance of your application, causing delays in processing messages and affecting overall throughput. By testing the resilience of your system to Kafka latency, you can identify any potential weaknesses in your system and take appropriate measures to improve its performance.

Structure

To conduct this experiment, we will ensure that all Kafka topics and producers are ready, and the consumer is receiving and processing messages correctly. We will then induce latency on Kafka by introducing a delay of XXX milliseconds on all incoming and outgoing messages. During the experiment, we will monitor the system to ensure that it continues to deliver its intended functionality and maintain its throughput despite the increased processing time. We will also analyze the monitoring data to identify any potential bottlenecks or inefficiencies in the system and take appropriate measures to address them. Once the experiment is complete, we will remove the latency and monitor the system's recovery time to ensure that it returns to its normal state promptly. By conducting this experiment, we can identify any potential weaknesses in our system's resilience to Kafka latency and take appropriate measures to improve its performance and reliability.