Liquid Cooling Pipeline Leakage Prediction and Detection using Reinforcement Learning

Over the years, hybrid and enterprise cloud adoption have grown significantly, data center server performance has improved continuously and edge computing infrastructure which is the spectrum between device and hyper-scale cloud have expanded dramatically. The explosive growth of these technologies have led to increase the cost of maintenance and energy consumption. To tackle the energy consumption problem, a cloud service provider has developed a smart cooling system since it is economical, environmentally friendly, provides better performance and low maintenance cost. However, when the Thermal Design Power (TDP) increases with increase in density, air cooling technology is not effective. Furthermore, a smart liquid cooling system can cover wide range of high-density computing products and can manage heat more efficiently than air cooling but deploying a liquid cooling system in the datacenter has critical challenges of safety and reliability. To address this problem, we propose Reinforcement Learning (RL) assisted methodology based on the contextual bandits approach and Markov Decision Process (MDP). The RL algorithm is composed of five components: Agent, State, Reward, Action, and Environment. The Agent (learner) repeatedly observes the liquid cooling system parameters (state) in real time, chooses an action and observes the reward for the chosen action. The agent trains on a wide range of simulated and physical data that represents the state of liquid cooling system (environment), and learns how to make decisions regarding to the state of the system, i.e., detect the leak.