Sign Language Recognition by LSTM-Driven Speech Synthesis

Humans interaction is done using either natural language like using words, writing and various gestures like hand gestures, head gestures and facial expression. But people with hearing impairments are not familiar with these languages. Thus, they use sign language for interacting with each other. Analyzing the natural language is easy as well as important whereas understanding gestures is also very important. For this reason, we have developed a sign language detection and recognition system which will help deaf and dumb people to communicate with others more efficiently. This is an innovative project/$stu$ dy to enhance communication for people who use sign language, especially with those with hearing impairments. The main objective of this system is to seamlessly detect sign language gestures, translate them into written text and then convert it into spoken language which elevates the communicative versatility of a system. The interpretation of sign provides proficiency to sign language users as well as to those who may not be aware of this language. The proposed system makes use of Machine learning and Deep Learning models, namely, MediaPipe Holistic and LSTM (Long Short-Term Memory) for feature extraction and sign recognition. The system achieves around 91.63% accuracy over testing data set. This project's significance extends beyond its technical capabilities. The deaf enhancement system is a valuable tool for people with hearing impairments which offers a promising solution for bridging communication gaps and promoting a more inclusive and accessible world.