Patch-based Markov random fields for fast face occlusion recovery

In this paper we present Markov random field (MRF) models for face occlusion detection and recovery. For occlusion detection, we use a pixel-based pair-wise MRF model (which is similar to the Ising model) where the binary mask on each pixel is inferred to decide the presence of occlusion. Then we construct a patch-based nonparametric pair-wise MRF model for occlusion recovery, which is learned using occlusion-free face images in the training set. Probabilistic inference using α-expansion leads to fast occlusion recovery, compared to the existing method. Numerical experiments confirm that our method speeds up the existing method by several orders of magnitude, while the quality of recovery is as good as the existing one.