Real-Time Sign Language Recognition with Deep Learning

This paper presents a real-time American Sign Language (ASL) recognition system based on deep learning (DL), aiming to improve accuracy and responsiveness over existing methods. A customized Convolutional Neural Network (CNN) was developed to recognize 36 static ASL gestures, covering digits (0–9) and letters (A-Z). A dataset consisting of 2500 images was constructed using a publicly available dataset from Kaggle and additional images collected independently to increase sample diversity. Data augmentation techniques such as scaling, translation, and rotation were applied to enhance generalization. The model's effectiveness was evaluated through five-fold cross-validation and compared against AlexNet, VGG16, and InceptionV3, with the proposed model achieving 94% accuracy on the test set. Real-time implementation was realized using OpenCV in the Visual Studio Code environment, with integrated background removal and image preprocessing to enhance robustness. The system accurately recognizes signs in live video streams with complex backgrounds and displays predictions in real time. This study demonstrates the potential of deep learning for accessible, real-time ASL interpretation. Future enhancements will target dynamic gesture recognition, expanded vocabulary, and deployment in real-world settings.