Static Alphabet American Sign Language Recognition using Convolutional Neural Networks

Sign language recognition is one of the vital research problems in computer vision systems. Sign language is used as a natural mode of communication by speech and hearing-impaired people in their daily life activities. This paper proposes a useful method to recognize static gestures with the application of Deep Learning. The pre-trained deep learning models trained on the ImageNet data and transfer learning technique are used to recognize 24 static alphabets of the American Sign Language (ASL) dataset. The publicly available benchmark American Fingerspelling dataset is used for the training of models. The paper presents the comparison of Convolutional Neural Network (CNN) models like VGG16, VGG19 and MobileNet on the same dataset and achieved average validation accuracy of 94.9% using the MobileNet model that gave good predictions using cross-validation. Also, the model has performed better than previous studies on the same dataset. The evaluation of the model performance is presented in the confusion matrix for the best trial fold that gives the prediction results of all 24 alphabets as recognition accuracy in percentage form.