Hyperparameter Optimization for Convolutional Neural Networks with Genetic Algorithms and Bayesian Optimization

Deep learning (DL) has emerged as a leading approach to solving complex problems in multiple areas. Its success is mainly due to technological development, large volumes of data, and various algorithms based on artificial neural networks (ANN). However, their functioning and performance are sensitive to the configuration of hyperparameters, which result from trial and error processes based on the knowledge and creativity of experts. In this sense, to reduce subjective associated with human intervention, an approach suggests tackling the tuning of hyper-parameters as an optimization problem. Here, the objective is to find a configuration that maximizes some performance metrics. The present work presents an implementation and comparison of three optimization strategies (Random Search, Bayesian Opti-mization, and Genetic Algorithm) for image classification in the CIFAR-10 dataset using convolutional neural networks, showing that algorithm performance and their solutions are tightly bound with the kind of architecture and search space.