Comparison of Handcrafted and Deep Neural Network Feature Extraction for Recognizing Bangla Sign Digits Exploiting Machine Learning Classifiers

Sign Language is the main communication medium for people who have a difficulty with speaking or hearing. To communicate with other deaf or healthy people, they use sign language. Deaf and mute people of Bangladesh and India use Bangla sign language to fulfill their communication needs. But, the learning process of sign language is time consuming and arduous. Therefore, as the artificial intelligence technology advances, automatic recognition of Bangla sign language is gaining more attention from researchers. In this paper, the proposed research aims to develop an approach to automate the identification of Bangla Sign Digits comparing the efficacy of handcrafted and deep neural network features. The handcrafted feature extractors like Histogram of Oriented Gradient (HOG), Scale invariant feature transform (SIFT), and pre-trained deep neural network feature extractors such as InceptionV3, Xception, and DenseNet-121 was used on publicly available Ishara-Lipi dataset to extract features. Moreover, to recognize sign digits, we used three fine-tuned classification algorithms namely K-Nearest Neighbor (KNN), Random Forest (RF), Support Vector Machine (SVM) that were fed by extracted features. Experimental result shows that the deep neural network features outperformed the handcrafted features based approach. DenseNet-121 combined with SVM based approach achieved highest test accuracy of 99.53% on Ishara-Lipi dataset.