Efficient Hyperparameter Optimization Using Deep Q-Network and BRKGA

Hyperparameter optimization (HPO) is paragon to maximize performance when designing machine learning models. Among different HPO methods, Genetic Algorithm (GA) based optimization is considered effective because it allows a wide and diverse range of solutions to be explored. However, GA's exploratory nature makes this type of algorithm to evaluate many solutions that do not improve the overall performance. This is specially costly when the objective function to be evaluated is time-consuming, like in the HPO field. In this paper, we propose an efficient hybrid algorithm that is able to reduce computational cost by combining deep reinforcement learning with the Biased Random Key Genetic Algorithm (BRKGA), a variant of genetic algorithms. Our reinforcement learning agent has a decision-making role during the population's fitness calculation, in which it filters out chromosomes that would not improve the overall fitness of the population. The agent uses small amounts of pre-trained data to identify trends in potentially good solutions, and carry out its decision process. We conduct experiments on eight different datasets to assess the effectiveness of the proposed method, and the results show that the proposed method can significantly reduce the computation time of hyperparameter search using BRKGA (up to 44% reduction in computational time) without compromising the quality of the solution (no statistically difference in results).