Detection of Sign Language Alphabets Using a Weighted Ensemble of Neural Networks

Speech disability is a condition that hinders a person's ability to communicate and express their thoughts through speech and hearing. Those with this impairment may use sign language as an alternative mode of communication. Despite the prevalence of sign language, non-signers may encounter difficulties in communicating with signers. However, recent advancements in computer vision and deep learning have greatly improved motion and gesture recognition. The main purpose of this project is to construct a deep learning application that can translate sign language into text, thereby facilitating communication between signers and non-signers. We have used three separate custom made CNNs (Convolutional Neural Networks), and then used them to create a weighted ensemble for detecting the sign language alphabets from a video frame. MNIST dataset is used to train the model here. The proposed CNN model provides the best accuracy of 96.23% in detecting the sign language alphabets. The system has effectiveness for recognition of hand signs.