Implementation of CNN-LSTM Integration for Advancing Human-Computer Dialogue through Precise Sign Language Gesture Interpretation

Sign gestures play a crucial role in communication for the deaf community, emphasizing the need for effective translation systems. Hand gestures convey thoughts more rapidly than other actions, making them essential for those who are deaf-mute. The primary goal is to design an assistive device that seamlessly translates a variety of sign signals into understandable text and vocalized output for individuals with speech difficulties. The proposed method involves creating a robust dataset with diverse hand gestures, employing image recognition preprocessing techniques, and implementing a CNN-LSTM model. This comprehensive framework combines computer vision, natural language processing, and speech synthesis for gesture recognition, text generation, and speech synthesis. Through extensive experimentation and model optimization, the system achieves a remarkable 94.5% accuracy in identifying and translating hand gestures into coherent text. Case tests with letters, numbers, and typical gestures demonstrate an average accuracy of 95%, showcasing the system’s versatility and durability.