Hard Samples Based Margin Loss for Face Verification

Although softmax loss and its variants have achieved great success in face verification, the performance is still subject to the data imbalance and early saturation problems. In this paper, we define hard samples as minority class samples and early saturation samples, in order to address both issues, we propose a new loss function termed Hard-Samples based Margin (HSM) loss. Inspired by the class-variant margin normalized softmax loss, we add larger margin on minority classes, the proposed real-class margin overcomes the negative influence from the data imbalance via making the optimization more balanced, while by expanding the margin of early saturated samples, the proposed pseudo-class margin keeps the samples away from the saturation region. Comprehensive experiments show that our HSM loss consistently surpasses the state-of-the-art loss functions on four popular face verification benchmarks.