Lip feature extraction for visual speech recognition using Hidden Markov Model

Visual speech recognition refers to recognizing the spoken words based on visual information of lip movements. In this paper, a new approach for lip reading is presented. Visual speech recognition is applied in science areas, such as speech recognition system and also in social activities, such as recognizing the spoken words of hearing impaired persons. The visual speech video is given as input to the face localization module for detecting the face region. The mouth region is determined relative to the face region. Different methods were used for feature extraction. Out of the different feature extraction methods, the 16 point DCT method gives the experimental results of 93.5% of performance accuracy. Then, these feature vectors are applied separately as inputs to the Hidden Markov Model (HMM) for recognizing the visual speech. 10 participants were uttered 35 different isolated words. For each word, 20 samples are collected for training and testing the HMM.