Real-Time Bidirectional Assistive Communication System for Speech and Hearing-Impaired Persons Using Deep Learning Models

Communication barriers create significant challenges for individuals who are deaf or mute, limiting their ability to interact effectively in daily life. This work presents a comprehensive assistive system that facilitates bidirectional communication by integrating speech-to-text, text-to-speech, sign-to-voice, and speech - to-sign language conversion using deep learning models. This proposed bidirectional assistive communication (BCA) system employs a microphone for speech input, which is converted into text using google application programming interfaces (APls) and displayed on a liquid crystal display (LCD) module. Similarly, sign language gestures are recognized via a camera, processed using a proposed modified deep learning mod-els (DLMs), and converted into synthesized speech. Additionally, the system introduces a Speech-to-Sign Language module, where spoken phrases are mapped to corresponding sign language GIFs or videos, enhancing accessibility for deaf individuals. This real-time bidirectional translation ensures a cost-effective, portable, and user-friendly solution for individuals with hearing and speech impairments. Experimental results indicate high recognition accuracy, low latency, and effective sign translation, demonstrating the system's potential to bridge communication gaps and promote inclusivity. Future enhancements include expanding the sign language dataset, improving gesture recognition across multiple languages, and optimizing real-time processing efficiency.