On the Implementation of a Software-Defined Memory Control Plane for Disaggregated Datacenters

By adopting a disaggregated hardware architecture, datacenters can achieve considerable efficiency gains and transition to a more sustainable and green future. By decoupling resources from a single monolithic server and connecting them through a high-speed optical network, it is possible to significantly increase resource utilization and reduce power consumption by consolidating workloads into fewer resource units. In this paper, we design and develop a software-defined control plane for disaggregated memory datacenters. Its core component, the Software-Defined Memory Controller, is the orchestrating software, which efficiently materializes the disaggregation concept. It accomplishes this by managing and monitoring remote resource pools, allocating resources to workloads, instrumenting the dynamic configuration of the underlying optical network for interconnecting remote compute and memory resources and interacting with software agents residing in host and guest Operating Systems for coordinating the attachment of remote memory. A major contribution of our design is the minimization of the delay for scheduling workloads and Virtual Machines in a disaggregated datacenter, which is accomplished with the efficient modelling of the disaggregated resources and networking elements into a graph for retrieving configuration data, as well as the optimization of the graph implementation. The architecture is based on the Everything-as-a-Service paradigm and is tightly coupled with OpenStack, the leading cloud infrastructure management software. Evaluation experiments validated the employment of a graph database system by the Software-Defined Memory Controller by demonstrating 58 percent faster query times than relational databases for a small-to-medium-sized datacenter, with the percentage increasing as the size of the datacenter grows.