Formation of Text from Indian Sign Language using Convolutional Neural Networks

Approximately 5% of people around the world suffer from a "disabling" hearing loss that will require rehabilitation. Around 2.42 million Deaf and Mute (D&M) individuals live in India, the world's second most populous country. In modern-day society, people do not think of these impairments as disabilities; it is merely perceived as a lifestyle choice. In reality, life is a lot harder for them as D&M individuals cannot communicate verbally, having to resort to mediums such as using visual media and finger spelling to communicate. However, the vast majority of non-D&M people are unaware of sign language, resulting in an extensive communication gap. This paper addresses this very issue by proposing a ML model that recognizes the Indian Sign Language (ISL) using Convolutional Neural Network (CNN) for sign language recognition and a Graphical User Interface (GUI) that captures these hand gestures and translates them into text. An ISL dataset with 31,945 images was used, and using the proposed approach an accuracy of 98.82% was achieved with the CNN model. This serves as the backend for a GUI which converts ISL to text.