IterDBR: Iterative Generative Dataset Bias Reduction Framework for NLU Models

Bias in natural language understanding (NLU) models often arises from reliance on spurious correlations in datasets. Existing dataset refinement methods frequently depend on predefined biased features, limiting their ability to address hidden biases. To address this, we introduce IterDBR, an Iterative Generative Dataset Bias Reduction Framework for NLU Models, which reduces dataset bias without the need for predefined features. Our approach utilizes an iteratively expanded sample pool where a shallow model assesses the bias levels of samples, leveraging large language models (LLMs) to generate pseudo samples with fewer bias. Various strategies, such as zero-shot prompting, few-shot prompting, and fine-tuning, were explored for optimizing the generator’s performance. Notably, combining these strategies improved the generator’s ability to create diverse and low-bias samples. Experimental results on three NLU tasks demonstrate that IterDBR surpasses state-of-the-art methods in performance and compatibility with model-centric techniques, showcasing its effectiveness in mitigating biases in diverse datasets.