Optimizing a Convolutional Neural Network using Particle Swarm Optimization

This study presents the application of the Particle Swarm Optimization (PSO) algorithm, a swarm algorithm which is based on the particle movement, to optimize the parameters of a Deep Neural Network (DNN), namely an architecture based on Convolutional Neural Networks (CNN). The model is optimized with respect to the image classification task on the MNIST dataset, consisting of images of handwritten digits. The study presents the results of training the model using different PSO hyperparameters and also compares the obtained performances with those obtained when training the model using gradient based optimizers such as Stochastic Gradient Descend (SGD) and Adam.