Video-Based Sign Language Translation System Using Machine Learning

The development of an interactive real-time video-based sign language translation system powered by efficient machine learning algorithms which is commonly developed for deaf-dumb people who are not able to hear or speak and is difficult for them to communicate among themselves or with normal people. Gesture and human activity recognition both are crucial for detecting the sign language as well as the behavior of an individual. These components are rapidly growing domains, enabling higher automation in households as well as in industries. Since extracting the features from continuous hand movements is complex and traditional sign language recognition gloves are costly, the combination of two deep learning algorithms, CNN and RNN can be used for automated sign language recognition. When both of these algorithms are used accuracy of the system also increases (the estimated accuracy is noted to be 92.4% on dynamic hand gestures studied on most of the available datasets). The system will then be able to translate the recognized sign language to desired text and then to speech for further communication using open-source Text-To-Speech API with python. This type of system has the potential in the future to enable a person to give the presentation or join a video conference in business or educational platform in which image or video-based representation of sign language can be projected as the person is speaking on a real-time basis. This architecture is well constructed and hence can solve many difficulties in communication purposes.