Neural regularization jointly involving neurons and connections for robust image classification

This paper presents an integrated neural regularization method in fully-connected neural networks that jointly combines the cutting edge of regularization techniques; Dropout [1] and DropConnect [2]. With a small number of data set, trained feed-forward networks tend to show poor prediction performance on test data which has never been introduced while training. In order to reduce the overfitting, regularization methods commonly use only a sparse subset of their inputs. While a fully-connected layer with Dropout takes account of a randomly selected subset of hidden neurons with some probability, a layer with DropConnect only keeps a randomly selected subset of connections between neurons. It has been reported that their performances are dependent on domains. Image classification results show that the integrated method provides more degrees of freedom to achieve robust image recognition in the test phase. The experimental analyses on CIFAR-10 and one-hand gesture dataset show that the method provides the opportunity to improve classification performance.