Increasing Energy Efficiency of Astrophysics Simulations Through GPU Frequency Scaling

The growing demand for high performance computing (HPC) necessitates significant energy consumption, posing a sustainability challenge for HPC centers, users, and society at large, especially in the face of stricter environmental regulations. While efforts exist to reduce overall system energy consumption, optimizations for GPU-based workloads, increasingly prevalent in HPC, has received insufficient attention for workload-specific energy efficiency optimizations. This work addresses this critical gap by proposing dynamic approaches to increase energy efficiency by instrumenting the simulation code with an open-source toolkit that enables accurate power and energy measurements for a wide range of CPU-GPU node architectures, as well as the instrumentation of the code controlling the GPU frequency dynamically. We further investigate the energy-performance trade-off by comparing both static and dynamic GPU frequency scaling strategies within SPH-EXA, a newly developed, open-source, and GPU-centric simulation framework specializing in astrophysical and cosmological simulations. Our findings demonstrate that code instrumentation enables detailed energy consumption acquisition beyond traditional HPC system monitoring, while dynamic frequency scaling of computational kernels achieves energy reduction with limited performance loss. This approach empowers researchers to develop more sustainable large-scale scientific simulations running mainly on GPUs.