SHDF - A Scalable Hierarchical Distributed Framework for Data Centre Management

A promising approach to increase the efficiency of infrastructure usage is to adapt the assignment of resources to workloads. This can be used, for example, to consolidate existing workloads so that the new capability can be used to serve new requests, or alternatively unused resources may be turned off to reduce energy consumption. Many architectural solutions have been presented for data centre management, however these tend to be centralised and may suffer in their ability to scale and support data centres with tens of thousands of nodes. Distributed approaches solve the scalability problem, however these do not have a global view of resources across the data centre. To address this, we propose a novel hybrid distributed hierarchical framework that is effective at providing the information needed for decision making at scale. We evaluate the performance of our approach by simulation, and demonstrate that a hybrid approach is a viable solution for managing large data centres, through rapid information dissemination and ability to make decisions using a global view.