Hand Gestures Translation System for Sign Language using CNN Models

Sign Language serves as an essential and dynamic medium of communication for individuals who are deaf or hard of hearing, offering a sophisticated system of gestures, body language, and facial expressions to convey ideas, emotions, and information. In response to this challenge, we introduce an innovative solution that harnesses the capabilities of neural network technology for real-time fingerspelling recognition in Sign Language. Our approach employs a multi-step process that begins with sophisticated filtering techniques to enhance the visual input of hand gestures. This refined data is then analysed by a neural network classifier trained specifically to recognize the intricate movements and configurations associated with fingerspelling in Sign Language. The paper aims to develop a sign language translator focusing on static hand gestures representing an alphabet. To address this, a lightweight Convolutional Neural Network (CNN) is developed. This CNN analyses static gestures captured through image input, recognizing and translating them into corresponding alphabet in text. By focusing on static gestures, the translator aims to provide efficient and accurate translations while minimizing computational complexity. This strategy will promote inclusivity and accessibility in a variety of contexts, including everyday encounters, education, and communication, by enabling smooth communication between sign language users and non-users. Remarkably, our system achieves an impressive validation accuracy of 97.33% through a CNN model VGG19 and training accuracy rate of 98.43% through ResNet50 model using MediaPipe across all letters of the Sign Language Alphabet. This technological breakthrough holds the promise of significantly reducing communication barriers and fostering greater inclusivity for individuals with hearing impairments.