Automated Scheduling Algorithm Selection in OpenMP

Scientific and data analysis applications are increasingly complex, with evolving computational and memory requirements during execution. Conversely, modern high performance computing (HPC) systems are heterogeneous and offer significant parallelism at the node and core levels. Scheduling and load balancing techniques are essential for maximizing the performance of applications on HPC systems. Recent work has shown the importance and the need of bringing scheduling techniques from the literature into commonly used parallelization frameworks, such as OpenMP. While this results in a multitude of scheduling options, it renders challenging the offline or online selection of the most suitable scheduling technique for an application-system pair, in the context of evolving applications’ computational requirements and variable capacities of modern HPC systems. Therefore, approaches for automatic selection of scheduling algorithms are urgently needed. This is an instance of the algorithm selection problem, proposed by Rice [1]. This oral communication presents recent results and ongoing work on the topic of automated scheduling algorithm selection for improving the performance of OpenMP applications. We present and evaluate two selection approaches, expert-based and reinforcement learning-based, both implemented in the LB4OMP scheduling library [2] – an extension of the LLVM OpenMP runtime library. The results show that automatic scheduling algorithm selection in LB4OMP surpasses manual selection, and that depending on the case, reinforcement learning-based selection outperforms expert-based selection or viceversa. This work is part of our ongoing efforts in solving the multilevel scheduling problem [3]. Similar scheduling algorithm selection solutions are needed at other parallelism levels, i.e., process level (MPI) and batch level (SLURM), to adapt to unpredictable variations both in applications and resources (including failures) that may arise during execution.