Fine-Tuning a Biased Model for Improving Fairness

Fairness has emerged as a crucial concern in machine learning since biased models would generate dissimilar predictions for different groups, perpetuating social inequalities. Although numerous techniques have been proposed to address the fairness issue in machine learning, most rely on incorporating fairness constraints during the training phase, rendering them ineffective once the model is deployed. This paper explores the potential of fine-tuning biased models to enhance fairness, particularly suitable for scenarios where retraining the model is not feasible. Our approach is rooted in an empirical analysis of the distribution of bias within a biased model, and we fine-tune the model parameter in a limited scope so that the performance of the original model can be maintained. We first observe that fine-tuning a biased model leads to deviations from its initial state, with deep layers undergoing the most significant changes. We then design and apply a bias-discovery algorithm, revealing that bias predominantly resides in the model’s deep layers. Based on these observations, we propose a straightforward yet highly effective method for debiasing the model: fine-tuning the classification head. We conduct a thorough theoretical analysis to justify the proposed method and provide guidance for fine-tuning. Furthermore, we experimentally validate our method on tabular and image datasets using four networks (CNN, AlexNet, VGG-11, and ResNet-18).