Understanding and boosting of deep convolutional neural network based on sample distribution

In order to improve the generalization ability of deep convolutional neural networks, an improved training strategy based on anomalous sample penalty term is introduced in this paper. We first establish an anomalous sample detection mechanism on the basis of preliminary neural network model, then we use the new loss function based on sample distribution to search an optimized feature boundary. A better boundary can improve the performance of the network over the test set. Then we discuss the effect of the convolutional neural network from the point of view of the sample distribution through a clean database created by ourselves. In the experiments, we compare the classification results of seven typical deep convolutional neural networks on different image databases. At the cost of reducing a little accuracy of training set, the classification accuracy of test set is improved significantly. We get the classification accuracy of 95.4%, 95.7% and 85.4% on Caltech-101, Cifar-10, Cifar-100 respectively. Finally, we analyze the influence of samples with anomalous distribution on network generalization capability through dimensionality reduction visualization.