Convolutional Neural Network-Based Natural Image and MRI Classification Using Gaussian Activated Parametric (GAP) Layer

We propose a novel gaussian activated parametric (GAP) layer for deep neural networks specifically for CNN. This layer normalizes the feature vector using the Gaussian filter-based un-sharpening technique. The goal of the proposed method is to normalize and activate the initial and intermediate feature layers of a deep CNN so that a customized layer can make the feature more distinguish and layers are smoothly tuned for the target-domain classification. Our experiment shows the use of the proposed layer for normalization gives a result almost similar to that of batch normalization and in few cases, the result is slightly better at the cost of higher training time. To demonstrate the result of the proposed layer we are using 4 layers of an encoder-based network as a base architecture for classification in which the first two normalization layers are GAP layers or BN layers whereas the remaining are BN layers. The result of the experiment is better in classification with two GAP layer as normalization layer in a bulkier dataset like ADNI MRI (3D CNN accuracy: 93.58% vs. 91.89%) and CIFAR-10 (2D CNN accuracy: 75.21% vs. 75.11%), whereas the result is better with replacement of first BN layer with GAP in a smaller dataset like 5-animals dataset (2D CNN accuracy: 62.92% vs 58.48%). Therefore, we suggest the use of single GAP layer normalization for smaller datasets and two GAP normalization layers for bigger dataset training. Also, the proposed method produced better results than the cross-channel normalization-based AlexNet network under scratch training.