An Improved Real-Time Sign Language Recognition using Transfer Learning

For decades, many researchers have been working on finding efficient solutions to implement sign language recognition systems through various technologies. Sign languages are visual languages that communicate through the hand, facial expressions, as well as body movements. Sign language is the bridge that connects us to the world of those who have impaired hearing or verbal ability. It allows them to understand the world around them through visual descriptions and, as a result, contribute to society. Thus, It strikes our mind to bridge the gap between the hearing impaired and normal people & make communication easier. Many researchers faced many challenges such as long training times to train a deep neural network model such as CNN or RNN, huge datasets of millions of images, high computational power requirements, the usage of expensive sensory devices, etc. Hence, the proposed method in this paper uses a technique known as “Transfer learning” where we use a high-accuracy pre-trained model and fine-tune its final layer parameters according to our task that is, Sign Language recognition which significantly reduces the training time and requires a small set of data of less than 20 images per gesture. Thus in this paper, we created a custom dataset generated using the system webcam, labeled data using the LabelImg tool, used Tensorflow Object detection API and downloaded a pre-trained model named “SSD MobinetNet V2” and applied Transfer Learning. Challenges faced in the project were issues with the correct labeling of images, a few installation errors, and lighting and background issues in a few detections. Overall, the method proposed is highly efficient in saving a lot of training time, providing high accuracy, and using fewer images for training.