Analyzing the Effects of Hyperparameters on Convolutional Neural Network & Finding the Optimal Solution with a Limited Dataset

Convolutional neural networks are composed of many hidden layers and each layer has its properties and may require some parameters. Such training models train on the weights derived from the training dataset. These weights determine how the input will have an impact on the output. The efficiency of the training model can be increased by fine-tuning the parameters of each layer. Manually tuned parameters that are known as hyperparameters require a heuristic approach for defining their values. In this paper, the hyperparameters, namely learning rate, filter size, number of filters, number of layers, and validation frequency, have been studied. This paper also examines the effect of the size of the dataset and the number of layers. It is observed that learning rate and number of layers have more impact on the accuracy as compared to validation frequency.