A Model-Agnostic Approach for Learning with Noisy Labels of Arbitrary Distributions

Most real-world datasets contain label noise, which can negatively affect downstream ML models trained on them. To deal with this problem, one can clean the mislabeled data before training, which is not only time-consuming and expensive but also requires domain expertise. Another approach is to use a noise-robust ML training algorithm. However, existing methods have some prerequisites that may not be practical in many applications (e.g., they are tied to specific downstream model architecture or they are applicable to specific noise distributions). In this paper, we propose a model-agnostic approach for learning with noisy labels of arbitrary distributions. In particular, our approach can work with any gradient descent optimization based machine learning model and deal with any label noise distribution. We achieve them by proposing two theoretically grounded noise-robust loss functions (for different noise distributions), and we are able to automatically decide which loss function to use based on a novel noise setting detection module. We directly learn the required hyper-parameters in the loss functions via meta-learning technique to minimize the loss on a given small clean validation set, and propose several strategies to improve the efficiency of training. Experiments on multiple datasets with both real-world and injected label noise show that our method performs better than state-of-the-art approaches.