Sign Language to Text Conversion in Real Time using Transfer Learning

The people in the world who are hearing impaired face many obstacles in communication and require an interpreter to comprehend what a person is saying. There has been constant scientific research and the existing models lack the ability to make accurate predictions. So we propose a deep learning model trained on the ASL i.e the American Sign Language which will take action in the form of American Sign Language as input and translate it into text. To achieve the former a Convolution Neural Network based VGG16 architecture is used as well as a Tensorflow model for image classification and we have improved the accuracy of the latter by over 4%. The dataset which has been used is the ASL dataset which has over 87000 images and has been used to train and test the video. There has been an improvement in accuracy from 94% of CNN to 98.7% by Transfer Learning. An application with the deep learning model integrated has also been built.