Label Noise Robust Curriculum for Deep Paraphrase Identification

In this paper, we study the effect of label noise on deep learning models for paraphrase identification. Curriculum learning, a learning paradigm that learns easy samples first and then gradually proceeds with hard ones, has shown excellent results in dealing with label noise for deep neural networks (DNNs). However, most previous studies focus on image classification, and design their curriculum only based on training losses of samples, ignoring domain-specific knowledge. In this paper, we propose a predefined curriculum learning based framework, incorporating both training losses of samples and domain-specific knowledge, to train robust deep models for paraphrase identification (PI) with label noise. Through extensive experiments on two popular PI benchmarks, we show that 1) the performance of the deep paraphrase identification model can drop sharply at the case of severe label noise; 2) our approach can significantly improve generalization performance of deep networks trained on corrupted data especially at extremely high levels of label noise; 3) our method can outperform several state-of-the-art label corruption robust methods.