Lipreading using a comparative machine learning approach

Lipreading is the process of interpreting spoken word by observing lip movement. It plays a vital role in human communication and speech understanding, especially for hearing-impaired individuals. Automated lipreading approaches have recently been used in such applications as biometric identification, silent dictation, forensic analysis of surveillance camera capture, and communication with autonomous vehicles. However, lipreading is a difficult process that poses several challenges to human- and machine-based approaches alike. This is due to the large number of phonemes in human language that are visually represented by a smaller number of lip movements (visemes). Consequently, the same viseme may be used to represent several phonemes, which confuses any lipreader. In this paper, we present a detailed study of the machine learning approach for the real-time visual recognition of spoken words. Our focus on real-time performance is motivated by the recent trend of using lipreading in autonomous vehicles. In this paper, machine learning approaches are applied to recognize lip-reading and nine different classifiers has been implemented and tested, reporting their confusion matrices among different groups of words. The classification process went on more than one classifier but these three classifiers got the best results which are GradientBoosting, Support Vector Machine(SVM) and logistic regression with results 64.7%, 63.5% and 59.4% respectively.