Indian Sign Language Interpretation using Skeletal Features and LSTM Networks

Indian Sign Language (ISL) is a standardized and commonly used medium of communication for the deaf and dumb community in India. Despite an enormous deaf and dumb community, there are only about 300 certified ISL interpreters. This necessitates robust automated sign language interpretation systems. For the classification of gestures, we develop LSTM and Bi-LSTM networks trained on skeletal features extracted via MediaPipe. Augmentation techniques are used to enhance model performance. Furthermore, a uniform frame sampling method is applied to reduce computational complexity and overfitting. We achieve an accuracy of 97.92% with the proposed system, thus demonstrating its high usability.