Group-Teaching: Learning Robust CNNs From Extremely Noisy Labels

Deep convolutional neural networks have achieved tremendous success in a variety of applications across many disciplines. However, their superior performance relies on correctly annotated large-scale datasets. It is very expensive and time-consuming to get the annotated large-scale datasets, especially in the medical field. While collecting a large amount of data is relatively easy, given the amount of data available on the web, but these data are highly unreliable, and they often include a massive amount of noisy labels. The past research works have shown that these noisy labels could significantly affect the performance of the deep convolutional neural networks on image classification. However, training a robust deep convolutional neural network with extremely noisy labels is a very challenging task. Inspired by the co-teaching concept, this paper proposes a novel method for training a robust convolutional neural network with extremely noisy labels, which is called group-teaching. Specifically, we train a group of convolutional neural networks simultaneously, and let them teach each other by selecting possibly clean samples for each network in each mini-batch. Each network back propagates the samples selected by other networks except itself and then it updates itself. The empirical results on noisy versions of CIFAR-10 and CIFAR-100 datasets demonstrate that our method is superior to the state-of-the-art methods in the robustness for noisy labels. Particularly, to verify the efficacy of our group-teaching in real-world noisy labels distribution, we have also validated the effectiveness of our method on the real-world noisy WebVision1000-100 dataset. The results show that our method has achieved higher performance than the state-of-the-art methods.