American Sign Language Alphabets Recognition using Hand Crafted and Deep Learning Features

Sign language is evolving as an inevitable communication method for the hearing impaired persons. The basic element of the sign language is the sign language alphabets. In this paper, a combination of hand crafted features and deep learning method is used to classify the signs in a more accurate manner. The skin color based YCbCr segmentation method and local binary pattern is applied for accurate shape segmentation and for texture features or local shape information. The transfer learning framework (VGG-19) is fine-tuned to obtain the features that are then fused with hand crafted features by serial based fusion technique. Finally, these features are given to a SVM classifier to classify the signs.