Real-Time Sign Language Interpretation: Integrating Gesture Recognition with Syntactic Analysis

This paper presents the development of an advanced system that has the capability to recognize hand gestures in real-time by utilizing state of the art libraries such as OpenCv and MediaPipe to convert sign language into text and artificial intelligence models. A new syntactic model has been implemented and also integrated into the proposed system to ensure high accuracy in gesture identification and also grammatically accurate sentences generated from gestures or from keyboard. Dynamic and static images are captured by the system through a simple web camera that extracts key features of the hands, wrists, joint and the degree of freedom. All gestures al labeled and categorized in three different datasets integrated into a user-friendly interface, Tkinter. Final results prove the systems robustness, accuracy, F1-score under different environmental conditions highlighting its potential to facilitate the communication with signers through sign language, especially with the users of the deaf-mute community.