Towards a Pre-Trained Question-Answering Language Model for Kinyarwanda

Kinyarwanda, as one of the official languages of Rwanda plays a significant role in the country's cultural, educational, and administrative domains. With the increasing demand for information retrieval and natural language understanding in Kinyarwanda, the development of robust Question Answering (QA) systems becomes imperative. Pretrained Language Models (PLMs) have demonstrated remarkable success in various natural language processing tasks, but their potential to address the specific challenges of Kinyarwanda QA remains largely unexplored. In this paper, we present a novel and comprehensive investigation into the feasibility and effectiveness of utilizing PLMs to tackle question-answering tasks for the Kinyarwanda language. We began by curating the annotated large-scale dataset of Kinyarwanda questions and corresponding answers, specifically tailored to the nuances of the language. The datasets utilized were from the sports and entertainment domain collected from several Kinyarwanda newspapers. Our approach relied on existing popular and robust Multi-Lingual language models. We developed various improved language models specific to Kinyarwanda QA tasks based on different models such as BERT, DistiIBERT, and Roberta. We assessed their performance on question understanding, context comprehension, and answer generation. Our findings have shown that the pre-trained language models exhibit promising potential in advancing Kinyarwanda OA with an F1 score of 50%.