Maximizing CNN Accuracy: A Bayesian Optimization Approach with Gaussian Processes

Convolutional neural networks (CNNs) are widely used in deep learning and the performance of their training process heavily depends on hyperparameters. Optimizing the hyperparameters can enhance the learning process and model performance. In this study, we utilized Gaussian Processes and Bayesian optimization techniques to explore the relationship between the performance of CNN models and their hyperparameters. Bayesian optimization is used to update the posterior distribution over the objective function using prior distribution information. We used the Expected Improvement (EI) acquisition function to select the next point that minimizes the objective function. To evaluate the effectiveness of our optimized models, we trained and tested them on standard benchmark datasets including MNIST, CIFAR-10, and SVHN. Our optimized models achieved high accuracy: 98% on MNIST, 90% on CIFAR-10, and 96% on SVHN, across a variety of CNN architectures.