Sign Vision: Real-Time Gesture Interpretation Powered by YOLOv11 Models

Speech impairments limit individuals’ ability to communicate verbally, often requiring sign language as an alternative communication medium. Bridging the communication gap between signers and non-signers remains a very critical challenge. Advances in real-time object detection algorithms, especially the YOLO family, have enabled more accurate and faster gesture recognition. This work proposes a real-time sign language recognition system that translates gestures into text using YOLO v11. Initially, CNN and Random Forest classifiers were explored; however, YOLO v11 was adopted due to its superior feature extraction capabilities and real-time processing performance. The system processes video input to detect and classify signs with high accuracy. Preliminary results show its effectiveness in recognizing static and dynamic gestures. The current work utilizes a substantially sized dataset comprising approximately 87,000 annotated images, capturing a range of sign gestures and variations in signing styles. This scale supports the model’s ability to generalize well across different signers and conditions. Future improvements may involve integration with sequence modeling techniques such as RNNs to capture gesture flow over time, further enhancing recognition performance. Overall, this research contributes to the development of accessible and scalable tools that help communication and inclusion for individuals with speech impairments.