Analysis of Cooling Water Temperature Impact on Computing Performance and Energy Consumption

The hot water cooling technique has been widely accepted as one of the standard techniques to improve the energy efficiency for the HPC and Data Centers. However, the higher operating temperature may impact the CPU power consumption due to the leakage current. Moreover, it may degrade the computational performance due to the DVFS mechanism activated to maintain the power and temperature within the TDP limit. In that sense, to fairly evaluate the efficiency of the hot water cooling technique, it becomes important to take into consideration not only the energy reduction on the HPC facility side (cooling system) but also the impact on the power consumption and the performance degradation on the HPC system side. In this paper, we utilized the Oakforest-PACS system and its facility, jointly administrated by University of Tsukuba and The University of Tokyo, in order to execute a quantitative and systematic analysis on the impact of the cooling water temperature onto the HPC system and its facility. For this purpose, we utilized lower (9 °C) and higher (18 °C) cooling water temperature other than the regular operational temperature (12 °C). Contrary to the gain in the energy consumption, on the HPC facility side, when using higher cooling water temperature, we observed an increase in the number of nodes suffering from performance degradation on the HPC system side. As a result, it can directly increase the probability of including low-performance nodes on multiple node jobs, and thus affecting their overall performance, especially during barrier synchronizations.