DRISHYAM: An Interpreter for Deaf and Mute using Single Shot Detector Model

There have been extensive studies in several fields that indicate hearing impairment and inability to verbally communicate lead to inequality of opportunity and even difficulties in everyday life. Despite being a very useful medium of communication for deaf and mute people, sign language has no meaning for someone who does not understand it. Identifying these hand gestures can be accomplished in two ways. One method of identifying is by static images, and another is by dynamic gestures. Upon reviewing the previous research, many limitations became apparent. In the existing system, high accuracy rates could be obtained when static images were fed, but accuracy rates decreased significantly when dynamic input was fed. With dynamic input, some systems achieved good accuracy rates, but their scope was inadequate. Hence as a solution to these difficulties, a system is proposed which acts as a medium for the deaf and mute communities to communicate with ordinary people who are unable to understand sign language. As an input method, the system offers two options. First, video frames are retrieved from the camera, and then processed and passed through the model to generate the text output. Second, the input is fetched as a word from a naive user without any prior knowledge of sign language. Using the segmented words from the database, the system will search for relevant video frames matching the segmented words and combine these video frames to produce a single video feed as an output. Thus the communication gap between the deaf-mute and a person with normal comprehension will be reduced.