Optimization algorithm behavior modeling: A study on the traveling salesman problem

Assume that there was an accurate model which can predict the solution quality that an optimization algorithm will reach if granted a certain amount of runtime on a given problem. Immediately, several applications in benchmarking, performance comparison, algorithm selection, algorithm analysis, and algorithm comparison arise. In this paper, two ways to obtain such models are introduced. The first one is by fitting nonlinear curves with fixed semantics and the second one is by training artificial neural networks on benchmark results. To validate the effectiveness of the models obtained by the two ways, a variety of use cases are investigated, including: 1) the interpretation of fitted curves in terms of their parameters using their fixed semantics in order to better understand the algorithm performance and problem hardness, 2) the classification of behavior data to algorithms, i.e., the detection of which algorithm was used to solve a given problem just according to its runtime behavior, 3) the prediction of how an algorithm may perform on a yet-unseen problem instance with yet-unseen features, based on its performance on other problems, and 4) the prediction of future algorithm performance based on models fitted to an (earlier) subset of the measured data. We present these use cases in a case study on the Traveling Salesman Problem. The experiment results show that all use cases are viable, easy-to-realize, and reliable.