Real-Time Sign Language to Speech Converter Using OpenCV and MediaPipe

Communication barriers between deaf individuals and hearing people can lead to several significant problems such as social integration, reduced access to information and reduced opportunities thereby affecting quality of life of deaf individuals. Use of sign language for communication serves as the primary means to address these challenges. However, understanding of the sign language is very difficult for hearing individuals. To address this issue, we present a Real-Time Sign Language to text and Speech Converter to facilitate effective communication between deaf and hearing individuals. This paper presents a novel approach to sign language recognition using OpenCV and MediaPipe. The main aim of the paper is to develop real time computer vision that can translate sign language gestures into text in real-time. We use image processing techniques to segment and analyze sign gestures, OpenCV to record and process these gestures, MediaPipe for hand tracking and gesture prediction and a text-to-speech engine to translate the text into speech for auditory communication. Thus, this study offers a comprehensive sign language to speech converter system that has the potential to significantly improve communication between the deaf community and the hearing people.