Adaptive Thompson Sampling for hyper-heuristics

There is an interest in search algorithms capable of learning and adapting their behaviour while solving a given problem. A hyper-heuristic operates on a set of predefined heuristics and applies a machine learning technique to predict which heuristic is the most effective to apply at a given point in time. Thompson Sampling is a machine learning mechanism interacting with the search environment to adapt its behaviour through trial-and-error. Despite the fact that it originated in the 1930s, the work on Thompson Sampling in the literature on search heuristics is limited. This paper is the first study investigating the Thompson Sampling approach in the field of hyper-heuristics. I propose an adaptive Thompson Sampling mechanism for hyper-heuristics and extensively evaluate its performance on a wide range of test models and combinatorial optimisation problems. The proposed algorithm is tested and compared with a large number of hyper-heuristics within a well-known competition for hyper-heuristics called CHeSC 2011. The results reveal that the proposed hyper-heuristic outperforms all the competing hyper-heuristics, including the state-of-the-art algorithm, on three combinatorial optimisation problems: (1) Personnel Scheduling; (2) Permutation Flow-shop, and (3) the Travelling Salesman problem.