Intent

Kubernetes features a rolling update strategy to deploy new releases without downtime. When being under load this only works reliably when your load balancer and the Kubernetes readiness probe are configured properly and DNS caches are up-to-date.

Motivation

The Kubernetes rolling update strategy ensures that during the deployment of a new release a minimum set of pods remain available. This implies that a new pod with a new release is started and needs to be ready before an old pod is evicted. Even so, this process may result in degraded performance and user-facing errors in the case, e.g., Kubernetes is sending requests to pods indicated as ready but not able to respond properly or evicted pods are still retained in the load balancer.

Structure

Before performing the rolling update all desirable pods of the deployment need to be in the “ready”-state, and a load-balanced user-facing HTTP endpoint is expected to respond successfully while being under load. As soon as the rolling update takes place the HTTP endpoint under load may suffer from a degraded performance (e.g. lower success rate or higher response time). Even so, this should be within the boundaries of your SLA. After the rolling update, the number of desirable pods matches the actual pods of the deployment and the performance of the user-facing HTTP endpoint is similar to before the update.

Environment Example

The Kubernetes deployment gateway consists of two pods and exposes an HTTP endpoint. We validate whether this HTTP endpoint is working with a success rate of at least 95% while updating both pods.

Solution Sketch

• Kubernetes liveness, readiness, and startup probes
• Kubernetes deployment strategy