Margin-Based Sample Filtering for Image Classification Using Convolutional Neural Networks

Deep convolutional neural networks have become the state of the art methods for image classification after demonstrating very good performance on very large datasets with general visual content. Amongst the problems for training deep CNN architectures is the heavy computational cost and the large memory requirements. In this work we exploit the fact that many training samples are correctly classified in the early stages of learning and therefore they can be skipped in further training epochs without noticeable performance degradation. We employ sample filtering to determine which samples are used in each epoch. We introduce the “multi-class margin” index to measure how safely a sample is classified to the correct class. The multi-class margin is closely connected with the output of the final Softmax layer. Using filtering we gradually reduce the number of usable samples to 40% of the original dataset. At the same time we gain around 18% in time compared to using all the samples in every epoch, while the performance degradation is around 1%.