Adversarial Embedding and Variational Aggregation for Video Face Recognition

Video face recognition is a challenging recognition problem due to low-quality and redundant video data. In order to efficiently address the problem, this paper proposes an adversarial embedding and variational aggregation (AEVA) approach that takes discriminative information from adversarial learning to implicitly constrain the distributions of video features. AEVA contains two parts: adversarial embedding learning and variational aggregation learning. The former contributes to discriminative feature embedding space by a self-adversarial process. It reduces intra-class distance of the same subject in an adversarial way. The latter aims to improve aggregated video representation by forcing latent feature distribution to be as close to real feature distribution as possible. An attentional weighting network and a variational inference structure are used to aggregate the features from one video and generate the latent features. Both parts have neither complex sampling strategies nor the hyperparameter settings from deep metric learning. Our approach achieves state-of-the-art performance for video face recognition on four widely used benchmarks, including YouTubeFace, IJB-A, YouTube Celebrities and Celebrity-1000.