Compressive Sensing based Predictive Online Scheduling with Task Colocation in Cloud Data Center

With the growing size of the cloud data center, the high scheduling efficiency over massive-scale cloud servers is hard to achieve, particularly when the scheduler requires the full real-time cloud resource information for decision making. Moreover, most data centers only run latency-critical online services, resulting in low resource utilization. To solve these problems, we propose a Compressive Sensing based Predictive Online Scheduling (CSPOS) algorithm. To mitigate the bottleneck of transferring massive resource information of all cloud servers to the scheduler, we propose to transfer sampled data from a small subset of servers to the scheduler and recover the full cloud resource information by compressive sensing. We then propose a predictive online learning algorithm that efficiently colocates the online services and batch jobs, in order to boost the resource utilization of the data center. Our experiments show that the CSPOS model achieves outstanding scheduling efficiency under various settings and is able to greatly increase the resource usage of a data center. We also illustrate that the running time of the CSPOS model is very small and has negligible effects on the scheduling system.