Real-Time Translation of Sign Language for Speech Impaired

Sign language is a visual language that uses hand gestures, change of hand shape, and tracking information to express meaning, and is the main communication tool for people with hearing and language impairment. Given the barriers faced by speech-impaired individuals, this system introduced a tool that bridges communication gaps and supports better interactions. This work focuses on introducing a tool that should bridge the communication gap among speech impaired community. The work involves the development of a system that enables two-way conversation between people with speech disorders and noisy people. LSTM networks were studied and implemented for the classification of gesture data because of their ability to learn long-term dependencies. In real-time, the sign language gestures of speech-impaired individuals are fed to the system by the device's computer vision capabilities. These gestures are recognized using deep neural networks, while hand recognition is cracked with edge detection algorithms that interpret in both text and speech formats. The model is trained with the dataset that is collected using holistic key points from the video of the person which detects the pose, face and hand landmarks. These will convert speech to text and finally displays the relevant hand gestures. This model can predict with an accuracy of 90%, showing the feasibility of using LSTM-based neural networks for the purpose of sign language translation.