Identify the effect of an unavailable third-party service on your deployment's service's success metrics.

Motivation

When you provide a synchronous service via HTTP that requires the availability of other upstream third-party services, you absolutely should check how your service behaves in case the third-party service is unavailable. Also, you want to validate whether your service is working again as soon as the third-party service is working again.

Structure

We ensure that a load-balanced user-facing endpoint fully works while having all pods ready. When we simulate the third-party service being unavailable, we expect the user-facing endpoint to work within specified HTTP success rates.. To simulate the unavailability, we can block the traffic to the third-party service on the client side using its hostname. The endpoint should recover automatically once the third-party service is reachable again.