Debiasing Low-Resource Language Models Via Cross-Lingual Transfer Learning

Large language models (LLMs) often amplify societal stereotypes present in training data. While several methods have been developed to debias models in English, less attention has been given to low-resource languages like Maltese and Urdu. This paper evaluates the effectiveness of multilingual models in transferring learned unbiased behavior across languages. We fine-tune a $\mathbf{X}$ LM-R m odel u sing t he W inoBias a nd StereoSet datasets in English and evaluate its bias across seven languages using the Multilingual CrowS Pairs Dataset. We find that our fine-tuned m odel r educes b ias b y 8% a cross 71 anguages, with a significant $\mathbf{1 9. 4 \%}$ r eduction inlow-resource 1 anguages like Maltese.