Real Time Sign Language Recognition Framework For Two Way Communication

The need to express oneself clearly and freely is a fundamental desire and must be available to all regardless of any handicap they face. Often, people that endure hearing and speech impairments use sign language as their only way of communication. This leaves a major communication barrier between people who speak sign language and those that communicate otherwise. Not a lot of people who communicate via vocal languages are inclined to learn sign language. The contribution of this paper is to develop a model which converts signs from the American Sign Language (ASL) into English characters using a CNN which segments the signs to their English language alphabets. The model was able to detect 26 alphabets and 3 additional characters with upto 99.78% accuracy. This paper also proposes inclusion of a speech to text module in order to facilitate seamless two way communication.