Real-Time American Sign Language Recognition Using Machine Learning

Sign Language Recognition (SLR) is a technology that enables computers to detect hands and classify hand gestures, facilitating an alternative form of human-computer interaction for individuals who use sign language to communicate. The MediaPipe framework is used in this project to achieve a high-accuracy responsive gesture detection system. This system is designed using SLR technology to detect American Sign Language (ASL). Researched methods for SLR vary in terms of cost and recognition accuracy, with recent advancements in Artificial Intelligence (AI) enable reliable implementation at a lower cost. Both equipment cost and algorithmic performance are considerations when developing an SLR system. The system must be practical to use in a real-world setting and capable of quickly recognizing gestures in a live environment. This project utilizes the MediaPipe task models, specifically the “Gesture Recognition Task” model, for tracking the subject’s hand and a custom model is created to detect ASL alphabet gestures. The application achieved an 83% accuracy with 26 classes and around 400 samples per class with real-time detection of hand gestures. Each class represents a different hand gesture, corresponding to a letter in the alphabet in ASL, and a ‘none’ class for when no hand gesture is detected.