Hierarchical Orchestration of Disaggregated Memory

This article presents XMemPod, a hierarchical disaggregated memory orchestration system. XMemPod virtualizes cluster wide memory to scale large memory workloads in virtualized clouds. It makes three novel contributions: (1) XMemPod offers efficient, transparent, and dynamic sharing of available memory that is disaggregated across VMs on the same host or in the cluster. (2) XMemPod provides a hierarchical memory expansion framework, which enables memory-intensive workloads on a VM to expand its memory demand over virtualized host memory first, and remote memory next, before resorting to external disk. (3) XMemPod provides a suite of optimization techniques to further improve the utilization and access latency of disaggregated memory. XMemPod is deployed on a virtualized RDMA cluster without any modifications to user applications and the OSes. Evaluated with multiple workloads on unmodified Spark, Apache Hadoop, Memcached, Redis and VoltDB, using XMemPod, throughputs of these applications improve by 11× to 612× over conventional Linux, and by 1.7× to 14× over the existing representative remote memory paging systems, and yet the total amount of network traffic consumed by XMemPod is only 24 percent of the existing approaches.