Context-Aware Contribution Estimation for Feature Aggregation in Video Face Recognition

The difficulties in video-based face recognition, such as dramatic pose variations and low quality, can be alleviated by leveraging the rich complementary information between the frames. However, limited by the mini-batch training strategy, the current deep learning methods only utilizes the frames in each batch during training, which ignore the context of the entire video. In this paper, we propose a context-aware feature aggregation scheme to aggregate complementary information between different frames. Firstly, a two-branch structure is designed as the
C
ontext-aware feature
A
ggregation
N
etwork (
CAN
). Secondly, a context-aware training strategy using a context bank is proposed, which alleviates the limitation of mini-batch samples by using the context of the entire video or several images belonging to the same ID and thus achieves global contribution estimation result. Comparative studies on benchmark datasets, such as IJB-C, YouTube Face (YTF), PaSC and COX, confirm that the proposed approach can achieve state-of-the-art level. Meanwhile, qualitative analysis on Multi-PIE dataset indicates that the contribution learned by the CAN is reasonable and beneficial to video face recognition.