Design and implementation of a lip reading system in smart phone environment

This study proposes a real-time lip-reading method in smart phone environment. In smart-phone environment where the resources are limited compared to existing PC environment, it is hard to operate lip-reading in real-time. To solve this problem, this study proposes the lip area detection method and feature extraction method suitable for smart-phone environment. First, to find the accurate lip area, the face area is detected by means of face color information, and the eyes are located to detect the lip area with the geometrical relation. To extract the outstanding features of the lip area in terms of changes in light according to the changes of surroundings, histogram matching, lip folding, and RASTA filter are applied, and the extracted data of features by means of PCA are used in recognition. As a result of the experiment, the changes were recognized almost in real-time, and 30 words of 50 were recognized, which indicates 60% recognition rates.