CTC-CNN-Bidirectional LSTM based Lip Reading System

The CTC-CNN-Bidirectional LSTM based Lip Reading System is designed to address the problem of accurate lip-based speech recognition. To better comprehend spoken words from lip movements, this system integrates Convolutional 3D Neural Networks with Bidirectional Long Short-Term Memory (LSTM) architecture and Connectionist temporal classification (CTC). It integrates 3D CNN with Bidirectional LSTM to provide a comprehensive knowledge of temporal and spatial features of the lip movements. This innovation provides a comprehensive understanding of lip dynamics. This model manages to get an accuracy of only 15.8% word error rate and 6.2% character error rate on the test dataset with 92 epochs. In the real world, this technology has the potential to substantially enhance communication for the hearing impaired, as well as safety and user interfaces.