Recognition of sequence lip images and its application

It is important in the study of Chinese lipreading to recognize Chinese language mouth shape sequences. In this paper, an approach based on parameter sequences of the mouth shape model in binary images is put forward and realized, then we discuss several possible applications, especially in the case of integration of lipreading and speech. First of all, we preprocess the captured sequence of images, then make them binary with an automatic binary threshold algorithm, extract the contour of mouth shape, fitted with two parabolas. Finally, we recognize mouth shape with HMM models using the features from the parabolas. The results of the above indicate that the system is satisfactory.