Steadybit logoResilience Hub
Try SteadybitGitHub icon
Steadybit logoResilience Hub

Load Balancing of Kubernetes Deployment when One Pod Crashes

If configured properly, Kubernetes is able to detect a non-responding pod and tries to fix it by simply restarting the unresponsive pod. Even so, the exact configuration requires careful consideration to avoid killing your pods too early or flooding your cluster’s traffic with liveness probes.
Download now

Load Balancing of Kubernetes Deployment when One Pod Crashes

If configured properly, Kubernetes is able to detect a non-responding pod and tries to fix it by simply restarting the unresponsive pod. Even so, the exact configuration requires careful consideration to avoid killing your pods too early or flooding your cluster’s traffic with liveness probes.
Download now

Load Balancing of Kubernetes Deployment when One Pod Crashes

If configured properly, Kubernetes is able to detect a non-responding pod and tries to fix it by simply restarting the unresponsive pod. Even so, the exact configuration requires careful consideration to avoid killing your pods too early or flooding your cluster’s traffic with liveness probes.
Download now

Load Balancing of Kubernetes Deployment when One Pod Crashes

If configured properly, Kubernetes is able to detect a non-responding pod and tries to fix it by simply restarting the unresponsive pod. Even so, the exact configuration requires careful consideration to avoid killing your pods too early or flooding your cluster’s traffic with liveness probes.
Download now
Go back to list
The experiment editor showing the visual structure of the experiment.The experiment editor showing the visual structure of the experiment.

Intent

In case a pod becomes temporarily unavailable you want to ensure that Kubernetes is properly reacting excluding that pod from the Service and restarting it.

Motivation

If configured properly, Kubernetes is able to detect a non-responding pod and tries to fix it by simply restarting the unresponsive pod. Even so, the exact configuration requires careful consideration to avoid killing your pods too early or flooding your cluster’s traffic with liveness probes.

Structure

Before killing a container of a Kubernetes pod we verify that a load-balanced user-facing endpoint is working properly and all pods of a Kubernetes deployment are marked as ready. As soon as one container crashes, Kubernetes should detect the crashed container via a failing liveness probe and mark the related pod as not ready. Now, Kubernetes is expected to restart the container so that the pod becomes ready within a certain amount of time. The user-facing HTTP endpoint may suffer from degraded performance when being under load (e.g. lower success rate or higher response time). Even so, this is expected to be within the SLA boundaries.

Environment Example

The Kubernetes deployment gateway consists of two pods and exposes an HTTP endpoint. We validate whether this HTTP endpoint is working even when one pod crashes with a success rate of at least 95%.

Solution Sketch


Download now

.json (5 kB)

It's quick and easy

  1. 1.

    Download .json file
  2. 2.

    Upload it inside Steadybit
  3. 3.

    Start your experiment!
Screenshot showing the Steadybit UI elements to import the experiment.json file into the Steadybit platform.
Tags
Kubernetes
Redundancy
GitHub
steadybit/reliability-hub-db/tree/main/recipes/kubernetes-deployment.load-balancing-pod-crash
License
MIT
MaintainerSteadybit

Used Actions

See all
HTTP check periodically

Execute HTTP calls and verify responses periodically.

Check

Check

Start Using Steadybit Today

Get started with Steadybit, and you’ll get access to all of our features to discover the full power of Steadybit. Available for SaaS and on-prem!

Are you unsure where to begin?

No worries, our reliability experts are here to help: book a demo with them!

Steadybit logoResilience Hub
Try Steadybit
HubActionsTargetsAdviceExtensionsRecipes
© 2024 Steadybit GmbH. All rights reserved.
Twitter iconLinkedIn iconGitHub icon