SENYAS: A Filipino Sign Language Recognition System Using MediaPipe and CNN-LSTM

Filipino Sign Language (FSL) is an important mode of communication for the deaf community in the Philippines yet the shortage of qualified interpreters creates barriers to effective communication. Thus, in this work, we developed a web application using computer vision and deep learning to recognize Filipino Sign Language (FSL) in real-time. The application extracts key features from hand gestures using Mediapipe and utilizes a CNN-LSTM model, combining strengths of convolutional neural networks and long short-term memory networks, to achieve high accuracy in recognizing both static and dynamic FSL signs. We performed a comparative analysis between five distinct CNN-LSTM models, evaluating them based on their performance on prediction speed, accuracy, precision, recall, and F1-score. The best performing model was CNN-MP-LSTM which achieved 98% accuracy during training, and 86% accuracy in real-world testing. Ultimately, this application has the potential to significantly improve communication accessibility for the deaf community in the Philippines.