Bidirectional Sign Language Translation

Communication is a tedious task for people who have hearing and speech impairment. Such people are assisted by a sign language translator for communicating with people. This however, is a slow process of translation, and many a times, it leads to loss of context in conversations, further leading to lack of involvement in social events. This research work proposes a portable and 24x7 available system with support for bidirectional translation i.e. from sign language to speech and speech to sign language. The portability of this system is assured with the use of a smartphone application, which is constantly available with the user, with no requirement of additional hardware. The mobile application will give normal speech output as audio and text and sign language output as a 3D animated video sequence, with the help of Unity3D. With the help of Machine Learning, the system will utilize a pre-trained model made with a combination of Convolution and Recurrent Neural Networks.