Intelligent Rack-Level Cooling Management in Data Centers With Active Ventilation Tiles: A Deep Reinforcement Learning Approach

The raised-floor architecture is widely adopted in the current data center industry. Due to the structural design and workload imbalance, many server racks suffer from a nonuniform inlet temperature in raised-floor data centers. This compromises not only the cooling performance, but also the computing capacity and system reliability. In this article, we employ the active ventilation tiles (AVTs), i.e., ordinary ventilation tiles with attached fans, to enhance the local cold air delivery and improve the cooling performance. We propose an AVT control algorithm adapted from the recently developed deep reinforcement learning (DRL) techniques to tackle the complex data center environment and thermodynamics. Different from previous studies where the algorithm required extensive pretraining before deployment, our algorithm is fully online. In order to improve the sample efficiency and accelerate the learning speed, we integrate the Dyna architecture to take advantage of the experienced system transitions. We also leverage the idea of shared reward and fingerprint to encourage the cooperation for multi-AVT control. The performance of the proposed solution is then evaluated by deploying a prototype implementation in a production data center. Experimental results reveal that our solution significantly improves the rack inlet temperature distribution.