Evaluation of Neural Network Models for Performance Prediction of Scientific Applications

Performance prediction is an important and active research area. In particular, several research efforts have built empirical models using machine learning algorithms for performance prediction. These models enable us to understand the dependence on hardware components for algorithm execution, system's scaling capabilities, cross-platform prediction in multi-core systems, and many others. The user community can use this knowledge to select hardware configurations best suited for executing a given software. In recent times, neural network-based models are widely used to build empirical models that understand complex relations between independent and dependent variables of an unknown data set. This paper has studied one-layer and multi-layer neural network models for performance prediction of three algorithms with different computations and memory-access patterns. We have shown that the multi-layer model outperforms the one-layer model, especially for computationally intensive algorithms. We have also shown that computationally intensive algorithms having a higher variance in runtime due to manufacturer variability require a higher number of neurons for convergence than memory-intensive algorithms. Our multi-layer neural network with optimal configuration has a prediction accuracy of about 88% for computationally intensive algorithms and about 95% for the memory-intensive algorithm.