Linear Feature Source Prediction and Recombination Network for Noisy Label Learning

Collecting training data for deep models from the Internet is a common data acquisition approach. However, there are challenges in using these data directly, as they often contain inaccurate annotations. This situation has increased the attention and importance of noisy label learning, the process of training a deep model with unreliable annotations. The typical strategy in noisy label learning is to identify potential mislabeled samples and assign pseudo-labels generated by the network to them, replacing the original labels. However, existing methods encounter the following problems: 1) they typically do not evaluate the pseudo-labels and directly use all of them, and 2) empirical parameter settings are often dataset-specific. These shortcomings limit the application of these methods in real-world scenarios. In this paper, we propose the Linear Feature Source Prediction and Recombination Network (LFSPR), trying to solve the problem above by proposing a new pretext task. The pretext task is designed to build the linear connection between the high-dimensional feature and the low-dimensional feature. The source of the latter is regarded as the high-dimensional feature, which follows a non-linear head network to obtain the low-dimensional feature. The pretext task is designed in low-dimensional space by predicting the linear composition weights of the potential source. Based on the pretext task, our method can generate pseudo-labels for uncertain samples while dynamically evaluating and selecting them, rather than simply using all pseudo-labels or discarding a fixed proportion of pseudo-labels for a given dataset. To the best of our knowledge, this is the first approach in the noisy label learning domain to employ pretext task for the pseudo-labels generation, evaluation and selection. The experiments on CIFAR-10, CIFAR-100 and Clothing1M demonstrate the effectiveness of our method.