Bridging the Gap: Real-Time Detection and Interpretation of Sign Language Using Convolutional Neural Networks

Sign languages enable communication and interaction for speech-impaired and hearing-impaired individuals. However, their usage is often limited to situations where visual signs can be understood, posing challenges for widespread communication. In the digital era, written communication using standardized languages predominates, which are easily recognizable. Converting signed languages into a computationally interpretable format is complex due to their nuanced, 3-dimensional nature. This paper proposes a computer vision system based on a convolutional neural network (CNN) to detect and interpret manual elements in sign language, simplifying their representation. Real-time object detection models including YOLOv3, YOLOv3- Tiny, YOLOv4, YOLOv4- Tiny, and SSD MobileNet were trained and evaluated. The YOLOv4 ASL model achieved an accuracy of 98.20% at 31.71 frames per second. This approach bridges the gap between sign language and written characters, advancing sign language interpretation.