Semi real-time algorithm for posture estimation of the human face using a 3-D reference picture

Numerous research studies in recent years have been done on analyzing gestures and sounds from video, and using them for man-machine interfaces. Human posture estimation by conventional techniques uses only 2D (2D) images or 3D data. However, estimation at fine angles is difficult using the 2D images, while use of special photography equipment is necessary to gather 3D data. In this paper, a technique for performing facial posture estimation from 2D video using a 3D reference picture is proposed. By this technique, the features of a human face are extracted using a 3D form of the human face obtained beforehand with a range finder. The 3D reference picture after the feature extraction is accumulated as a database. The human facial posture in an animation is presumed in analyzing the 2D facial domain extracted from the accumulated 3D reference picture and the animation. In posture estimation processing, in order to make the size and inclination of a human face, calibration is performed on the human face domain extracted from the 2D input image using a circular mask. Furthermore, a movement vector is computed from the relation with the previous and subsequent processing frames, and human face operation in the processing frame is predicted based on the calculated result. From the prediction results, posture estimation processing is accelerated by thinning out the rotation angle of the 3D reference. The key feature of this technique is in its ability to perform posture estimation with robust sufficient accuracy to the size or inclination of a human face.