Analysis of Potential Online Scheduling Improvements by Real-Time Strategy Selection

Task Scheduling in large-scale HPC platforms is normally accomplished with simple heuristics combined with a backfilling algorithm. Some strategies, such as the First-Come-First-Served (FCFS) with backfilling, provide reasonable results in a variety of scenarios, including different HPC platforms and task set characteristics. But for each scenario, a different strategy might be the most appropriate for minimizing some metric, such the as the average task waiting time or turnaround time. In this work, we evaluate the effects of choosing different scheduling strategies over sub-sequences of workload logs of 6 real HPC platforms, for periods from 1 to 24 hours. For each platform and workload period, we show that the performance of each scheduling strategy have large variations for different workload sub-sequences. Similarly, the best scheduling strategy for each sub-sequence also varied. Finally, we show that, if one could select the best strategy for each workload sub-sequence, it would significantly reduce the scheduling performance variations and improve the mean queue waiting time by more than 50% for most cases. These results indicate that the development of heuristics or machine learning algorithms for selecting the best scheduling strategy every 6 or 24 hours, can result in significant improvements in the mean queue waiting time of tasks in HPC platforms.