Multistep Hyperparameter Tuning via Reinforcement Learning for Simulated Annealing

In this study, a hyperparameter (HP) tuning method for simulated annealing (SA) is proposed. In recent years, annealing machines, i.e., non-Neumann architecture computers inspired by the annealing process of spins in the Ising model, have been developed to solve various optimization problems quickly. The performance of annealing machines depends on various HPs, e.g., the temperature range and cooling rate, which are required to set the annealing conditions. Due to the commonality of optimization calculations based on the Ising model, HP tuning methods for SA can be applied to annealing machines. A representative HP tuning method is multistep tuning, where HPs are changed adaptively during the annealing steps. However, adaptive tuning tends to increase computation time costs; thus, the number of HP tuning cycles in multistep tuning methods should be minimized. Therefore, the proposed method implements reinforcement learning (RL) into multistep tuning to set HPs appropriately for various inputs while minimizing the number of tuning cycles. The proposed RL-based method was evaluated by introducing a benchmark environment, and the results demonstrate that the proposed method can obtain the optimal policy that provides appropriate HP values with fewer cycles compared to existing tuning methods.