Real-time Arabic Sign Language Recognition using CNN and OpenCV

Sign language is the way of interacting for hearing impaired people. Each Sign language has multiple dialects. The world suffers from a lack of interaction between impaired and hearing individuals. This work proposes an application that recognizes Arabic Sign Language (ArSL) using Convolutional Neural Network (CNN) and OpenCV. The CNN model is trained using ArSL2018 with over 5000 images with 32 categories. Next, the OpenCV is used to capture a real-time frame. After that, the CNN model and OpenCV method are combined to recognize 32 letters. The model shows a weighted-average F-measure of 95%. Similarly, the macro-average F-measure was 95%. Finally, the application is run and the prediction of the letters in real-time is successful.