Real-Time Multilingual Communication System Using CNNs for Bidirectional Sign and Speech Translation

A person who is proficient in Indian Sign Language cannot currently converse with someone who is proficient in sign languages from other countries. Additionally, there isn't an interface that translates between Indian Sign Language and other regional languages in India. The World Federation of the Deaf estimates that there are as many as seventy million people in the world using sign languages. It is the only way that the deaf people can express their feelings and emotions. Availing this language for the hearing and speech impaired is indeed of great significance to those who are learning as it shows the intent behind the expression and gives guidance on the person’s signing right away. The proposed solution aims to develop an interface that would close the communication gap that exists within the sign language community. Their ability and self-assurance in communicating with others through sign language can be enhanced by this. The system combines numerous machine learning models operating in the background with multiple translations, sign-to-sign, speech-to-sign, and sign-to-speech. These are employed to distinguish and identify the signs and convert them into the necessary spoken or sign language. These models are combined with a website that runs well to carry out the necessary translations. The system processes additional translation fast and effectively after receiving real-time video input. The goal of this research is to create a useful tool for converting between sign languages. Additionally, we may convert a specified sign language into a text or speech. translated from ASL to ISL with an 82% accuracy rate on average. To identify the indicators, our system receives video data in real-time. With our technology, a dumb or deaf person can converse with a blind person. Since our technology recognizes signs in real time, it may also be utilized by someone who wishes to learn sign language.