Intent

Verify that New Relic alerts you on disruptions in your New Relic workflow such as a critical deployment having no pods ready to serve traffic.

Motivation

Kubernetes features a liveness probe to determine whether your pod is healthy and able to accept traffic. If Kubernetes is unable to probe a pod, it is restarting it in the hope to get it ready eventually. In case it is a critical deployment, New Relic workflow should alert on this disrutpion

Structure

First, check that the New Relic Workflow is marked as operational As soon as all pods of a deployment aren't reachable, caused by the Steadybit attack block traffic, New Relic should detect this by marking the workflow as disrupted and ensuring your on-call team is taking action.

Environment Example

The Kubernetes deployment gateway consists of two pods. We are attacking both pods and wait for New Relic to detect the disrupted New Relic workflow.

Solution Sketch

• Kubernetes liveness, readiness, and startup probes
• New Relic Workflow