A Comprehensive Review of Recent Advances in Deep Neural Networks for Lipreading With Sign Language Recognition

Lip reading is a form of “listening” to people that happens visually. It’s also referred to as “Speech reading.” This is done by observing the speaker’s face and listening to the spoken words. The mechanism comprises face identification, lip localization, feature extraction, and finally identifying the phrase/sentence based on the lip movements. It recognizes the text through lip movements. The expert system has learned by transferring the end user-generated lip-movement frames with the sequence. It will distinguish the lip movement and the uttered word using visual input or audio and video data. This article discusses the improvement of lip reading and the involvement of sign language in lip reading. The feature extraction methods, lip-reading approaches, datasets, parameters, and different languages used are discussed in this article. At the same time, the article also focused on various datasets and approaches for integration of sign language with lip reading. Finally, the challenges of recent lipreading techniques and future research directions are highlighted.