Language-agnostic Zero-Shot Machine Translation with Language-specific Modeling

Zero-shot translation plays a key role in the multilingual Neural Machine Translation (NMT) domain, allowing multilingual systems to translate language pairs unseen in training. Despite the potential importance, it often suffers from poor translation performance. It is a challenge to achieve high translation quality when the models generalize to new translation directions. To address this issue, we break down the task of zero-shot translation into two parts: language-specific modelling and language-agnostic translation training. We integrate a large number of language pairs to train a general language-agnostic translation model to improve the zero-shot translation performance. Additionally, we introduce the language-specific models by denoising training to improve the performance on specific directions. Our experiments demonstrate that our approach significantly improve the zero-shot translation performance compared to baselines.