Lip motion capture and its application to 3-D molding

It is essential for machine lip reading to process not only static images but also moving images. This paper proposes a method, which is named Optical Snakes, for lip motion extraction from moving images using the cooperation between the active contour models called Snakes and the optical flow. The effectiveness of the method is demonstrated by the extraction of a series of lip contours from time-varying images without any markers, and by the analysis of lip contours for machine lip reading. Some real products which characterize a speaker's pronunciation are made from the lip motion data by a photoforming system. It is demonstrated that the real products are better than the virtual products on a computer display for the evaluation of shapes. The application of lip motion to molding would lead to a new method for the analysis of lip motion and the training of pronunciation.