Thoth: Provisioning Over-Committed Memory Resource with Differentiated QoS in Public Clouds

Due to workload fluctuation in virtualized systems running multiple Virtual Machines (VMs), tenants often allocate more memory than actually used to handle peak memory demands. Memory adjustment by reclaiming inactive memory in VMs is an effective way to enable memory overcommitment so as to improve memory utilization and reduce the cost. However, memory overcommitment may increase the chance of swapping and thus results in performance degradation. While prior work has proposed a variety of memory adjustment schemes to try to guarantee the performance of workloads, it still remains unexplored to reduce disparity in performance across co-located VMs during dynamic memory adjustment and investigate the chance of far memory support to release memory pressure by orchestrating rack-level stranded memory. To bridge the gap, we propose Thoth, a rack-level memory resource orchestrator that enables a fairness-aware memory adjustment scheme to guarantee differentiated performance across all co-located workloads within each physical server while opportunistically leveraging available far memory to reduce the performance degradation of workloads from traditional SSD- based swapping. Thoth adopts a credit-based resource allocation strategy cross all co-located VMs to maintain a long-term fairness and reduce disparity in performance. Thoth is evaluated with typical cloud applications in our data center environment. The results show that Thoth could improve the performance of latency-critical workloads by 3 x on average.