Grafana
Extension
Grafana
A Steadybit check implementation for data exposed through Grafana.Extension
Grafana
Extension
Grafana
A Steadybit check implementation for data exposed through Grafana.Extension
Go back to listYouTube content is not loaded by default for privacy reasons.
Introduction to the Grafana Extension
The Steadybit Grafana Extension bridges the world of Steadybit and Grafana. The extension adds checks to your Chaos Engineering experiments to validate Grafana Alert rules state and reports events of your experiments to Grafana as annotations to ease correlation.
Integration and Functionality
Integration of Grafana into Steadybit works via the Grafana API. Thus, all you need is a Grafana's Service Account Token.
Integration of Grafana in Steadybit
With the Alert Rule Check you can integrate your Grafana alert rules into your experiments. Verify that Grafana alert rules notice a fault injected by Steadybit to check that your observability strategy is working as expected.
Integration of Steadybit in Grafana
The extension automatically reports experiment executions to Grafana, which helps you to correlate experiments with your dashboards.
Installation and Setup
To integrate the Grafana extension with your environment, follow our setup guide.
Grafana alert rule fires when a Kubernetes pod is in crash loop
Verify that a Grafana alert rule alerts you when pods are not ready to accept traffic for a certain time.
Motivation
Kubernetes features a readiness probe to determine whether your pod is ready to accept traffic. If it isn't becoming ready, Kubernetes tries to solve it by restarting the underlying container and hoping to achieve its readiness eventually. If this isn't working, Kubernetes will eventually back off to restart the container, and the Kubernetes resource remains non-functional.
Structure
First, check that the Grafana alert rule responsible for tracking non-ready containers is in an 'okay' state. As soon as one of the containers is crash looping, caused by the crash loop attack, the Grafana alert rule should fire and escalate it to your on-call team.
Solution Sketch
- Kubernetes liveness, readiness, and startup probes
Grafana alert rules
Kubernetes cluster
Kubernetes pods
