LS-CNN: Characterizing Local Patches at Multiple Scales for Face Recognition

Faces in the wild may contain pose variations, age changes, and with different qualities which significantly enlarge the intra-class variations. Although great progresses have been made in face recognition, few existing works could learn local and multi-scale representations together. In this work, we propose a new model, called Local and multi-Scale Convolutional Neural Networks (LS-CNN). First, since similar discriminative face regions may occur at different scales, it is necessary to learn multi-scale features. To this aim, we introduce a new backbone network, namely Harmonious multi-Scale Network (HSNet), which extracts rich multi-scale features from two harmonious perspectives: utilization of different kernel sizes in a single layer, and concatenation of multi-scale feature maps from different layers. Second, identifying similar local patches is important when global face appearances have dramatic changes. Meanwhile, different face regions have different discriminative abilities. To capture critical local similarities and weigh adaptively on different local patches, a spatial attention is proposed. Third, channels have different convolutional kernels which can detect different features with various importance. Besides, hierarchical channels concatenated from different layers contain diverse information: channels from low layers describe local details or small-scale parts, and channels in high layers represent high-level abstraction or large-scale parts. To emphasize important channels and suppress less informative ones automatically, channel attention is used. Due to the complementary characteristics of channel attention and spatial attention, they are fused to form the Dual Face Attentions (DFA). To the best of our knowledge, this is the first effort to employ attentions for the general face recognition task. The LS-CNN is developed by incorporating DFA into HSNet model. Experimental results on various face matching tasks show its capability of learning complex data distributions.