Hybrid Algorithm for Real-Time Sign Language Detection System

This research paper suggests a machine learning-based system designed to interpret sign languages in real-time from a video feed to help deaf and hard of hearing individuals overcome communication barriers. Sign languages employ hand gestures, facial expressions, and body language to convey ideas and meaning visually. Unfortunately, individuals with hearing impairments face communication difficulties, limited access to information and services, and social isolation. The proposed system captures the subject's gestures using a camera, which are then processed using a TensorFlow object detection API that predicts based on a pre-trained machine learning model. The system's methodology incorporates supervised learning, Single Shot MultiBox Detector (SSD) object detection algorithm, and the luminosity method for converting colour images to grayscale. For dataset creation, hand gesture image samples were captured and labelled using open-source graphical annotation software LabelImg, and the model was trained. The system proposed achieved an average accuracy of 95% and has the potential to improve communication and reduce marginalization for deaf and hard of hearing individuals.