Low-Resource Language Summarization: A Study of Bangla Using T5 architecture

This paper presents the performance of mT5 and bengali-T5-base models on natural language understanding and summarization tasks, especially for the low-resource language Bangla. The training and validation loss trends have shown that bT5-base converges better, learns faster, and has lower final losses compared to mT5. This is further emphasized in the performance metrics, where bT5-base achieved a significantly higher Exact Match score of 0.1343 compared to mT5’s 0.0003. In this respect, bT5-base had the best results on ROUGE metrics, with an average ROUGE-1 F1 of 55.63%, whereas mT5 was very far behind, with an average of 2.49%. The generated summary test demonstrated how bT5-base could generate summaries that are correct and in line with the context, while mT5 generated totally irrelevant summaries. These results show how crucial architectural design and pretraining strategies have been for bT5-base in its efforts to outperform mT5 in capturing and generating relevant outputs. The results hint at some paths the improvement of mT5 should take by using either a hybrid approach or more significant fine-tuning to bridge the gap in the performance.