EXTES: An Execution-Time Estimation Scheme for Efficient Computational Science and Engineering Simulation via Machine Learning

In recent years, computational science and engineering (CSE) simulations using high-performance computing resources are actively exploited to solve complex domain-specific problems. Thanks to the remarkable advance of IT technology, the CSE community is challenging more complex and difficult problems than ever before, by running these simulations online. In this regard, we often witness that 1) online simulation users suffer from knowing little about the estimated termination time of their launched simulations and 2) the limited computing resources are squandered by wrong input that leads the simulations to run forever. To address such issues, we propose a novel execution time estimation scheme, termed
EXTES
, using machine learning techniques for more efficient online CSE simulations. With a large amount of existing provenance data, the
EXTES
scheme trains a suite of models rooted from classification, regression, and a hybrid of the two and utilize these models to estimate the execution time for specified input parameters for simulations. In the experiments on real simulation data, our proposed models achieved about 73% accuracy on average in execution time estimation across 16 simulation programs taken from a variety of CSE fields. In the meantime, the overhead incurred by the training and estimation is almost negligible.