An Improved Pooling Scheme for Convolutional Neural Networks

The study introduces a new pooling scheme called Accept-Reject Pooling which uses a stochastic procedure and integration of a rejection sampling based method of selection from a multinomial distribution of the pooling region's activations. The stochastic component of the proposed pooling operation ensures that non-maximal activations will have a chance to be selected and passed to the network while ensuring that the strong activations get the higher chance of being sampled. Experiments demonstrate that the approach can also work well with other techniques such as Batch Normalization and Data Augmentation Convolutional Neural Network achieve improved performance with Accept-Reject Pooling as compared to the use of several pooling methods such as max, stochastic and mixed pooling on benchmark image classification datasets.