Towards MPEG4 compatible face representation via hierarchical clustering-based facial feature extraction

Multi-view imaging and display systems has taken a divide and conquer approach to 3D sensing and visualization. We aim to make more reliable and robust automatic feature extraction and natural 3D feature construction from 2D features detected on a pair of frontal and profile view face images. We propose several heuristic algorithms to minimize possible errors introduced by prevalent imperfect orthogonal condition and non-coherent luminance trying to address the problems incurred with illumination discrepancies on common surface points in accommodation of multi-views. In our approach, we first extract the 2D features that are visible to both cameras in both views. Then, we estimate the coordinates of the features in the hidden profile view based on the visible features extracted in the two orthogonal views. After that, based on the coordinates of the extracted features, we deform a 3D generic model to perform the desired deformation based modeling. Finally, the face model is texture-mapped by projecting the input 2D images onto the vertices of the face model. As the reconstructed 3D face model is MPEG4 compliant, it can be readily animated by standard MPEG4 facial animation parameters (FAPs). Present study proves the scope of modeling procedure for real-time applications such as face recognition and performance driven facial animation.