Investigation on Immersion Cooling Solution for Hyper-scale Data Center Application

Novel technologies such as cloud computing, Big Data, Machine Learning (ML) and Artificial Intelligence (AI) have been widely applied worldwide to cater for various industrial application, eventually demanding more and more powerful data centers. Data center's thermal cooling strategy has been evolving in the past years and most traditional data centers still adopts air cooling solution for IT equipment's' cooling and dissipates the waste heat to environment directly. However, air cooling solution has almost reached the bottleneck to cool high power of CPU and GPU and also consumes substantial portion of total data center's power budget, which hurt both energy efficiency and operational expense (OpEx). As governments around the world have begun to request optimal Power Usage Effectiveness(PUE) for newly built large scale data centers, advanced liquid cooling solution is urgently being requested. One of the approaches is the immersion cooling solution with IT systems immersed in thermally conductive dielectric liquid, which has recently gained increasing attention from the industry to address the challenges in terms of thermal design and PUE requirement. However, due to lack of published validation result and long-term reliability statistics of immersion cooling solution, most data centers still hesitate to adopt this novel cooling technology. This paper introduces the best practice of single-phase immersion cooling solution applied by Volcano Engine, the cloud service platform under ByteDance.COM, including the architecture of immersion cooled data center, liquid tank, liquid candidates and server systems as well as thermal cooling performance, signal integrity impact, and results comparison with air cooled servers. The material compatibility and long-term reliability is excluded from this paper as related testing is still in progress and not concluded yet.