Imbalance-Aware Adaptive Margin Loss for Fair Multi-Label Face Attribute Recognition

This paper proposes a novel metric learning method for fair face attribute recognition. Datasets with imbalanced sample distributions can spoil the fair discrimination ability of deep learning models. This problem becomes more complex in multi-label classification tasks due to the variety of imbalance levels for each label. To tackle this problem, we propose Imbalance-Aware Adaptive Margin (IAAM). IAAM utilizes prior information on data imbalance with new parameters that control the relative margins between classes for each label. Specifically, it adaptively tunes margins in the metric learning loss to give larger margins for minority classes and smaller margins for majority classes. The proposed method is a flexible reformulation of previous studies for multi-label classification tasks. We performed experiments on the Celeb-A and LFWA datasets and evaluated with fair classification metrics. Our findings showed that the IAAM led to excellent performance on fair face attribute recognition both in the weakly imbalanced labels and strongly imbalanced labels.