Block Traffic
Attack
Containers
Block Traffic
Attack
Containers
Go back to list
Kubernetes deployment survives Redis downtime
Check that your application gracefully handles a Redis cache downtime and continues to deliver its intended functionality. The cache downtime may be caused by an unavailable Redis instance or a complete cluster.
Motivation
Redis downtime can lead to degraded system performance, lost data, and potentially long system recovery times. By testing your system's resilience to Redis downtime, you can ensure that it can handle the outage gracefully and continue to deliver its intended functionality. Additionally, you can identify any potential weaknesses in your system and take appropriate measures to improve its performance and resilience.
Structure
We will verify that a load-balanced user-facing endpoint fully works while having all pods ready. As soon as we simulate Redis downtime, we expect the system to indicate unavailability appropriately and maintain its throughput. We can block the traffic to the Redis instance to simulate downtime. The experiment aims to ensure that your system can gracefully handle the outage and continue delivering its intended functionality. The performance should return to normal after the Redis instance is available again.
Containers
Datadog monitors
Kubernetes cluster
Kubernetes deployments
Graceful degradation when Microsoft SQL Server database can not be reached
An unavailable Microsoft SQL Server database might be too severe for suitable fallbacks and requires your system to indicate unavailability appropriately
Motivation
Depending on your context, an unavailable Microsoft SQL Server database may be considered so severe that there are no suitable fallbacks. In this case, ensuring that your system indicates an appropriate error message is essential. After the Microsoft SQL Server database returns, your system should recover automatically.
Structure
We ensure that a load-balanced user-facing endpoint fully works while having all pods ready. When we simulate an unavailable Microsoft SQL Server database, we expect the user-facing endpoint to indicate unavailability by responding with a "Service unavailable" status. To simulate the unavailability, we can block the Microsoft SQL Server database client connection on its hostname so that no incoming or outgoing traffic goes through. The endpoint should recover automatically once the Microsoft SQL Server database is reachable again.
Containers
Kubernetes cluster
Kubernetes deployments
Graceful degradation when Oracle database can not be reached
An unavailable Oracle database might be too severe for suitable fallbacks and requires your system to indicate unavailability appropriately
Motivation
Depending on your context, an unavailable Oracle database may be considered so severe that there are no suitable fallbacks. In this case, ensuring that your system indicates an appropriate error message is essential. After the Oracle database returns, your system should recover automatically.
Structure
We ensure that a load-balanced user-facing endpoint fully works while having all pods ready. When we simulate an unavailable Oracle database, we expect the user-facing endpoint to indicate unavailability by responding with a "Service unavailable" status. To simulate the unavailability, we can block the Oracle database client connection on its hostname so that no incoming or outgoing traffic goes through. The endpoint should recover automatically once the Oracle database is reachable again.
Containers
Kubernetes cluster
Kubernetes deployments
Graceful degradation when Postgres database can not be reached
An unavailable Postgres database might be too severe for suitable fallbacks and requires your system to indicate unavailability appropriately
Motivation
Depending on your context, an unavailable Postgres database may be considered so severe that there are no suitable fallbacks. In this case, ensuring that your system indicates an appropriate error message is essential. After the Postgres database returns, your system should recover automatically.
Structure
We ensure that a load-balanced user-facing endpoint fully works while having all pods ready. When we simulate an unavailable Postgres database, we expect the user-facing endpoint to indicate unavailability by responding with a "Service unavailable" status. To simulate the unavailability, we can block the Postgres database client connection on its hostname so that no incoming or outgoing traffic goes through. The endpoint should recover automatically once the Postgres database is reachable again.
Containers
Kubernetes cluster
Kubernetes deployments
