Improving Low-Resource Question Answering with Cross-Lingual Data Augmentation Strategies

We investigate cross-lingual data augmentation to improve question answering (QA) systems in low-resource settings. Specifically, we propose 2 strategies in fine-tuning a multilingual pretrained language model (PLM) with cross-lingual QA data. The first is the combined approach, a strategy that simply combines QA data in 2 different languages, that is, a low-resource language as the intended fine-tuning target, and a higher-resource language for augmentation, to be used simultaneously during fine-tuning. Our second strategy is 2-step fine-tuning, where we first fine-tune the multilingual model with a QA data from the augmentation language, then fine-tune the resulted model with the data of the intended target language. Our experiments showed that the 2-step fine-tuning strategy boosts the final QA model in the target language when compared to vanilla one-step fine-tuning using only the target language as training data. Our experiments are mainly focused on Indonesian as the target language, using the FacQA dataset, while we use various English QA datasets for augmentation. We further show that our approach is extensible to QA in other languages. All codes and data used are available at: https://github.com/ryanpram/2step/finetuning/xlda