Self-Updating with Facial Trajectories for Video-to-Video Face Recognition

For applications of face recognition (FR) in video surveillance, it is often costly or unfeasible to collect several high quality reference samples a priori to design representative facial models. Moreover, changes in capture conditions and human physiology create divergence between facial models and input captures. Multiple classifier systems (MCS) have been successfully applied to video-to-video FR, where the face of each individual of interest is modeled using an ensemble of 2-class classifiers (trained on target vs. non-target samples). However, the reliable self-update of these individual-specific ensembles with relevant target and non-target samples raises several challenges. In this paper, an adaptive MCS is proposed that allows for self-updating facial models given face trajectories captured during operations. Different faces appearing in a camera viewpoint are tracked, and ensemble predictions for facial captures are accumulated along each track for robust video-to-video FR. When the number of positive predictions over time surpasses an update threshold, the target face samples extracted from the trajectory are combined with non-target samples selected from the cohort and universal models for efficient self-update the corresponding face model. A learn-and-combine strategy is then employed to avoid knowledge corruption during self-update of an ensemble. At a transaction level, the adaptive MCS outperforms the reference systems that do not allow self-updating on Face in Action videos. Analysis at a trajectory level indicates that the proposed system allows for robust spatio-temporal recognition, which translates to enhanced security and situation analysis.