Combat Noisy Labels by Joint Training

Learning with noisy labels is one of the most challenging problems in WSL. Classical supervised learning assumes that models are trained with instances from clean data distribution. But in real world, instances with noisy labels are ubiquitous. With the development of Deep Learning, researchers find that Deep Neural Networks (DNN) are prone to overfit noisy instances gradually due to memorization effects. To tackle this problem, many seminal works in label-noise representation learning (LNRL) have been proposed to improve the performance of Deep Learning models. In this paper, we proposed a new learning paradigm via the lens of optimization. Specifically, we train two identical networks and each network utilizes small-loss policy to select reliable instances from mini-batch data. And then we take the union of selected instances from two networks as training examples. From the perspective of supervised information, we argue that our proposed approach can augment the supervision of LNRL via two networks' diversity. Empirical results on noisy version of MNIST, CIFAR-10 and CIFAR-100 demonstrate that our approach is superior towards other state-of-the-art approaches in LNRL and can effectively avoid overfitting noisy labels.