LIP Reading for Specially Abled Persons

Lip Reading is also called Visual Speech Recognition, is a technique that utilizes only the visual scene of speech recognition. The proposed framework deals with three main steps: Detecting human faces and lips, feature extraction, and classification. The novel method used in this work is the detection of salience in images. To deal with the temporal data's the image frames were concatenated. The proposed work uses a public database called MIRACL-VCl that includes different words and phrases. The work focuses on the English word recognition based on the Convolutional Neural Network (CNN) with and without salient features. Both networks used the VGG 19 pre-trained network and fine-tuned the layers to make it suitable for the dataset mentioned. Salient detection of image frames improves the accuracy of the system than the normal flow.