Mixture Statistic Metric Learning for Robust Human Action and Expression Recognition

Background objects and textures in real-world video sequences often pose great challenges for human action and facial expression recognition. This paper proposes a mixture statistic metric learning for recognizing human actions and facial expressions in realistic “in the wild” scenarios. In the proposed method, multiple statistics, including temporal means and covariance matrices, as well as parameters of spatial Gaussian mixture distributions, are explicitly mapped to or generated on symmetric positive definite Riemannian manifolds. An implicit mixture of Mahalanobis metrics is learned from the Riemannian manifolds. The learned metrics place similar pairs in local neighborhoods and dissimilar pairs in relatively orthogonal regions on a regularized manifold. The proposed metric learning method also explores the prior distributions within the multiple statistics in the video sequences. The proposed method is tested on five action video data sets and three facial expression data sets and is compared with various state-of-the-art methods. Recognition accuracy and computational efficiency are evaluated in terms of average recognition rates and computational times in seconds, respectively. Competitive performances achieved on both action and facial expression recognition tasks demonstrate the effectiveness of the proposed method.