CARE: A Cost-AwaRe Eviction Strategy for Improving Throughput in Cloud Environments

To utilize various resources efficiently, cloud data centers usually deploy Latency-Critical (LC) and Best-Effort (BE) applications as containers in their physical machines by assigning higher priorities for LC jobs to use the resources. Due to the increase on the workload, it needs evicting some BE jobs to deprive more resources for LC jobs to meet the Quality of Service (QoS) requirements of LC jobs. Under dynamic workload settings, the frequent BE job eviction results in significant throughput degradation. To this end, we propose a novel Cost-AwaRe Eviction strategy, CARE, which takes into account both the recalculation cost and the remaining time cost of each BE job. CARE fully exploits the two types of different costs of BE jobs and chooses a BE job with the lowest overhead for eviction under different resource demands of LC jobs. Furthermore, CARE improves the throughput of BE jobs without affecting the performance of LC jobs. We prototype and implement CARE atop Kubernetes. Experimental results show that CARE achieves up to 1.70 × throughput gain compared to state-of-the-arts while its negative impact on the performance of LC jobs is negligible.