A SFM-based sparse to dense 3D face reconstruction method robust to feature tracking errors

In this paper, we present a new sparse to dense 3D face reconstruction method using monocular video sequences. Structure from motion (SFM) is an effective method to reconstruct sparse 3D facial shape; however, its performance degrades drastically when tracking errors caused by self-occlusion or image noise exist. To address the problem, we propose a reliable point selection method to automatically evaluate the reliability of corresponding points obtained by optical flow. The gray level cooccurrence matrix (GLCM) is applied to the texture-based correlation evaluation and those points whose correlation coefficients are lower than a threshold will be removed. Benefiting from the SFM's capacity of dealing with missing data, our method is more robust to tracking point correspondence errors and accordingly achieves a lower 3D reconstruction error compared with traditional SFM methods without correlation checking.