New Relic Detects a Kubernetes Pod in Crash Loop
New Relic Detects a Kubernetes Pod in Crash Loop
When one of your containers has problems starting, it may result in a crash loop and, eventually, Kubernetes backing off to restart this container. Verify that New Relic notices a crash loop and will alert you to take action.New Relic Detects a Kubernetes Pod in Crash Loop
New Relic Detects a Kubernetes Pod in Crash Loop
When one of your containers has problems starting, it may result in a crash loop and, eventually, Kubernetes backing off to restart this container. Verify that New Relic notices a crash loop and will alert you to take action.Intent
Verify that New Relic alerts you on pods not being ready to accept traffic for a certain amount of time.
Motivation
Kubernetes features a readiness probe to determine whether your pod is ready to accept traffic. If it isn't becoming ready, Kubernetes tries to solve it by restarting the underlying container and hoping to achieve its readiness eventually. If this isn't working, Kubernetes will eventually back off to restart the container, and the Kubernetes resource remains non-functional.
Structure
First, check that New Relic doesn't have any critical events for related entities. As soon as one of the containers is crash looping, caused by the Steadybit attack crash loop, New Relic should detect this via an incident to ensure your on-call team is taking action.
Environment Example
The Kubernetes deployment checkout
consists of two pods.
We are attacking one of the two pods by causing a crash loop and waiting for New Relic to detect the incident of the crashing pod.
Solution Sketch
Download now
.json (3 kB)
It's quick and easy
- Download .json file
1.
- Upload it inside Steadybit
2.
- Start your experiment!
3.