Enhancing Intrinsic Features for Debiasing via Investigating Class-Discerning Common Attributes in Bias-Contrastive Pair

In the image classification task, deep neural networks frequently rely on bias attributes that are spuriously cor-related with a target class in the presence of dataset bias, resulting in degraded performance when applied to data without bias attributes. The task of debiasing aims to compel classifiers to learn intrinsic attributes that inher-ently define a target class rather than focusing on bias at-tributes. While recent approaches mainly focus on empha-sizing the learning of data samples without bias attributes (i.e., bias-conflicting samples) compared to samples with bias attributes (i.e., bias-aligned samples), they fall short of directly guiding models where to focus for learning in-trinsic features. To address this limitation, this paper pro-poses a method that provides the model with explicit spa-tial guidance that indicates the region of intrinsic features. We first identify the intrinsic features by investigating the class-discerning common features between a bias-aligned (BA) sample and a bias-conflicting (BC) sample (i.e., bias-contrastive pair). Next, we enhance the intrinsic features in the BA sample that are relatively under-exploited for pre-diction compared to the BC sample. To construct the bias-contrastive pair without using bias information, we intro-duce a bias-negative score that distinguishes BC samples from BA samples employing a biased model. The experi-ments demonstrate that our method achieves state-of-the-art performance on synthetic and real-world datasets with various levels of bias severity.