MixLoss: A Data Augmentation Method for Sequence Generation Tasks

In this paper, we propose MixLoss, a simple and effective data augmentation method for sequence generation tasks like machine translation, speech recognition, etc. MixLoss works by mixing two inputs and two losses with two corresponding outputs using the same weights, which can also be applied in the hidden space. Concretely, when the loss takes the form that we agreed, we prove that MixLoss is theoretically equivalent to Mixup and thus has an adequate theoretical foundation. We corroborate our theoretical analysis with experiments that support our conclusions. Experimental results on the EnglishGermany dataset reveal that MixLoss achieves the best performance compared with other data augmentation methods, and can integrate well with other technologies. Specifically, the BLEU can be improved from 27.10 to 28.41 on the MT task, and the WER can be decreased from 18.9% to 15.1% on the ASR task. Moreover, MixLoss has stronger robustness to resist noisy data.