Recognizing Arabic Sign Language gestures using depth sensors and a KSVM classifier

The objective of this research is to detect, recognize and interpret hand gestures of Arabic Sign Language (ArSL) letters using two sensor devices - Microsoft's Kinect and a Leap Motion Controller (LMC). The image data captured of the body and the hands' skeletal structure were processed using a supervised learning approach and natural user interface libraries. This research developed a model to examine 1400 signs gestured by 20 users for 28 ArSL letters. The depth images of the hand captured by the sensors extracted 77 angles for each joint and 26 angles between each two bones. To overcome the challenge of time complexity of capturing and interpreting data for this model, the Principle Component Analysis (PCA) algorithm was used for simplification of the large dataset by deleting redundant, irrelevant or erroneous data due to noise. By using PCA, the 103 captured data for each gestured letter was reduced to 36 which is enough to provide more than 99% variance of the data. Then by implementing the Kernel Support Vector Machine (KSVM) classifier on the remaining dataset, the overall accuracy results for the training data increased to approximately 93% using the SVM algorithm, while the accuracy for identifying the corresponding Arabic sign language letters from the test data was at 86%.