Fill the container's ephemeral storage with random data for a given duration.
Please note that a container always shares resources with all other containers running on the same host. Although the attack executes in the container's cgroup and namespace, it might affect the available host's resources and, thus, other containers running on that host as well.
Use Cases
Test the container's or cluster's behavior when the file system has no free space left
Validate the container's lifecycle when exceeding Ephemeral Storage limits
Parameters
Parameter
Description
Default
Duration
How long should the disk be filled?
30s
Mode
Decide how to specify the amount to fill the disk: overall percentage of filled disk space in percent, Megabytes to write, Megabytes to leave free on disk
overall percentage of filled disk space in percent (PERCENT)
Size
Depending on the mode, specify the percentage of filled disk space or the number of Megabytes to be written or left free.
80
Path
Path where the file should be written
/tmp
Method
Should the disk be filled all at once (using fallocate) or gradually over time (using dd)?
at once (AT_ONCE)
Block Size
Only applicable for method over time (dd): The block size when writing the file. If the block size is larger than the size value, the attack uses the size value as the block size.
5
Frequently Asked Questions (FAQs)
Q: What is the difference between the available modes?
A: The disk fill attack differentiates between two modes (at once and over time). Read further to understand the difference and which is the best choice for you.
Mode 'at once' The mode is perfect if you want to check how your system behaves when the disk is almost full, and you don't need to simulate a gradual decrease in free space.
However, if you want to write more than 100% of the disk space, the gradual approach (see below) might be a better fit as it works more reliably.
The mode uses the fallocate Linux command, which allocates the disk space without writing actual data and, thus, results in the disk filling up very fast.
Mode 'over time' The over-time approach is perfect when you want to simulate a more realistic case of a gradually filling disk or need to fill the disk more than available space.
The mode uses the dd command, which writes actual data (zeros) to the disk and is thus slower than the at-once approach (see above).
Q: How is the number of free megabytes on disk calculated?
A: The attack uses the following formula to ensure that the specified megabytes are left free on the device (defined by the size parameter):
diskSpace.Available - size * 1024
Where
diskSpace is the output of df -kP <filepath>
size is the amount of megabytes to leave free on disk
Q: How is the percentage of filled disk space calculated?
A: The attack uses the following formula to calculate how many kilobytes need to be written based on the configured percentage (defined by the size parameter):
diskSpace.Capacity * size / 100 - diskSpace.Used
Where
diskSpace is the output of df -kP <filepath>
size is the configured percentage of the disk space to be writen
Q: How can I check whether the disk is full?
A: Use the command df -h to check if the disk is full.
Q: Why is my disk not filled up to the specified value/percentage even though the attack was executed without errors?
A: The attack calculates the amount to write only once at the beginning of the attack using the formulas mentioned above. If other processes cause the used disk space or overall capacity to vary while running the attack, you may have a higher or lower filled disk percentage. Also, the 'over time' approach works more reliably if you need to write more to the disk than available space.
For an elastic and resilient cloud infrastructure, ensure that the over-usage of ephemeral storage of one container doesn't affect any others.
Furthermore, if one container exceeds its configured limits, Kubernetes should kill it and eventually prepare it within a given timeframe.
Structure
Verify that a user-visible endpoint responds within the expected success rates while exceeding the ephemeral storage.
As soon as one container exceeds the ephemeral storage, by filling the disk in a /tmp directory, Kubernetes should evict the container, decreasing ready pods. Within 60 seconds, the evicted container should run again, and the pod should be ready.