Deep Learning Framework for Fingerspelling System using CNN

Sign language is used for communication among the hearing-challenged-cum-speech-challenged community. It is the language that uses facial expressions, movements of hands and other body parts. Indian Sign language (ISL) is commonly used sign language, in India. These days, online interpreters are available for translating the sign language to common language and vice versa. But it requires an expert who can translate in both ways and also they charge. Thus, the communication between the hearing-challenged-cum-speech-challenged community and the rest has become difficult and costly. This causes isolation of the hearing-challenged-cum-speech-challenged community from the rest of society. Hence, this work focuses on bridging the communication gap between the two. One important aspect of sign language is finger spelling. Through finger spelling, a word that lacks a particular sign in sign language can be expressed by spelling each alphabet of that word. Here, a vision based finger spelling system using Convolutional Neural Networks (CNN) is created. The proposed CNN model performs as par with the InceptionV3 and better than ResNet and VGG16 models.