A Data Driven Approach for Computer Vision based Sign Language Model using Convolutional Neural Network

Human beings communicate with each other with the help of speech. However, not everyone on our planet is endowed with the ability to talk and hear. People with such difficulties in speech and hearing, cannot communicate with each other by normal means. So, they use set of signs and hand gestures to communicate with each other. The set of such hand signs and gestures is known as sign language. Yet, they cannot communicate with normal people as most people out there do not know how to use sign language. The available methods are applied only on the level of alphabets, making them useless for live time usage. The primary goal of this research work is to convert sign language into speech on world level with the help of computer vision, tensorflow and natural language processing by extracting features from hand signs along the features from facial emotions as facial emotions also play vital role in the world level sign language. The hand signs and gestures along with the facial emotion features are converted into text with the help of an object detection model supported by tensorflow and computer vision. The artificial intelligence techniques - convolution neural network has been used for the training and predicting the speech input feed. The text output from the object detection model is then converted into speech with the help of pyttsx package, which is a renowned python package in the field of natural language processing.