A Multi-Armed Bandit selection strategy for Hyper-heuristics

Meta-heuristics have emerged as an efficient way to solve NP-hard problems even without the guaranteed of optimal values. The main issue of meta-heuristics is that they are built using domain-specific knowledge. Therefore, they require a great effort to be adapted to a new domain. The concept of Hyper-heuristic was proposed to solve this problem. Hyper-heuristics are search methods that aim to solve optimization problems by selecting or generating heuristics. Selection hyper-heuristics choose from a pool of heuristics a good one to be applied at the current stage of the optimization process. Although there are several works focused on selection hyper-heuristics, there is no consensus about which is the best way to define a selection strategy. In this work, a deterministic selection strategy based on the concepts of the Multi-Armed Bandit (MAB) problem is proposed for combinatorial optimization. Multi-armed bandit approaches define a selection function with two components; the first is based on the performance of an operator and the second based on the number of times that the operator was used. In this work, three MAB algorithms were implemented using the HyFlex framework. An empirical parameter configuration was performed to each algorithm, and the best setup was compared to the top ten CHeSC 2011 algorithms using the same methodology adopted during the competition. The results obtained were comparable to those attained by the literature. Moreover, it was concluded that the behavior of MAB selection is heavily affected by its parameters. As this is not a desirable behavior for hyper-heuristics, future research will investigate ways to better deal with the parameter setting.