CNN-Based Real-Time ASL Gesture Recognition and Communication Aid

Sign language is an essential communication tool for deaf and hard-of-hearing people, conveying ideas through systematic hand movements. Yet, the inability of non-signers to identify sign language leads to continuing social and practical issues. In this paper, a novel real-time system is presented for the purpose of recognizing American Sign Language (ASL) finger-spelling hand gestures and translating them into text and sound outputs, ensuring inclusivity and accessibility. By integrating MediaPipe for precise hand landmark detection with a Convolutional Neural Network (CNN) for efficient classification, the system has a simulated accuracy of 98.8% in controlled experiments. It records gestures through a webcam, processes them to remove ambient noise, and uses ‘pyttsx3‘ to translate text to speech, thus providing a seamless interface for non-signers. Leverage of Google Colab’s GPU support ensures quick training and consistent performance. Exhaustive measurement of parameters like accuracy, inference time, and reliability shows its efficacy in different settings like education, healthcare, and social interaction, thus bridging communication gaps in deaf communities effectively.