Batch Normalization in Convolutional Neural Networks — A comparative study with CIFAR-10 data

Deep learning is an emerging field of computational science that involves large quantity of data for training a model. In this paper, we have performed a comparative study of various state-of-the-art Convolutional Networks viz. DenseNet, VGG, Inception (v3) Network and Residual Network with different activation function, and demonstrate the importance of Batch Normalization. It is shown that Batch Normalization is not only important in improving the performance of the neural networks, but are essential for being able to train a deep convolutional networks. In this work state-ofthe-art convolutional neural networks viz. DenseNet, VGG, Residual Network and Inception (v3) Network are compared on a standard dataset, CIFAR-10 with batch normalization for 200 epochs. The conventional RELU activation results in accuracy of 82.68%, 88.79%, 81.01%, and 84.92% respectively. With ELU activation Residual and VGG Networks' performance increases to 84.59% and 89.91%, this is highly significant.