Indian Sign Language Phrase Estimation Using PoseNet

Communication between people is very important to convey thoughts and ideas. The predominant mode of human communication is speech. But often people with communication and hearing impairments are not able to communicate properly with other people. The most popular form of communication for these individuals is sign language. Although there are at least 15 million deaf signers who use Indian Sign Language (ISL), there aren't enough technologies developed to interpret ISL and facilitate communication for signers who use it. This calls for the development of more resources for creating awareness and providing easy and accessible communication to deaf people. The aim of this paper is to aid in this initiative. In this work we use a webcam to capture the sign shown by the person, identify them using a pre-trained machine learning model PoseNet and classify them with ml5.neuralnetwork with an average accuracy of 92.85%.