Conversion of LIP movement to speech: An aid to physically impaired and dumb people

Silent Speech Interface(SSI) has given rise the possibility of speech processing, even inthe absence of an acoustic signal and it has been used as an aid for the dumb person or speech handicapped. A Silent Speech Interface is a device that enables the communication to take place when voice is not present. In noisy environments, the SSI holds a promising approach for speech processing. In this project a system is developed which recognizes silently uttered words and performs specific tasks based on the recognized lip movements of the words. The method used is noninvasive. The system records video of the lip movements of the words/letter uttered silently. The recorded signal is analysed by extracting features using local binary pattern (LBP) operator and then detected using a kNN classifier. The system is trained and tested to recognize the word ‘WATER’. When the word ‘WATER’ is uttered, lip movements are analyzed and the control signal is generated to initiate the robot to pick a water bottle from a specified location and place it at the predefined destination.