Improving Runtime Prediction Performance Based on Application Type of Jobs on Large-scale Cluster System

Despite the exponential increase in computing resources over the past few decades, the demand for computing resources continues to overwhelm supply due to the emergence of services requiring large-scale computing resources, such as ChatGPT. Under this circumstance, various technologies are required not only to continuously expand the scale of computing resources but also to improve their efficiency. To efficiently utilize resources, various backfilling techniques have been developed and applied, and accurate job runtime prediction is essential for this.This study presents an analysis of job runtimes based on job logs collected from a national leadership computer. First, the results of a correlation analysis between job features and job runtimes are presented, and the possibility of time-series prediction is confirmed through stationary characteristic analysis. Additionally, the study examines how including the type of application as a dependent variable for runtime prediction can improve prediction performance. The experimental results show that including the type of application improves the performance of all three machine learning techniques tested in the study. In particular, the performance of the random forest method improved by 61% and 74% for RMSE and MAE, respectively.