Real-Time Kannada Sign Language Recognition to Speech and Text for Rural Primary Healthcare Centers

Understanding and interpreting sign languages are crucial for promoting communication among the Deaf and Hard of Hearing (DHH) populations [1]. Kannada Sign Language (KSL), primarily utilized in Karnataka, lacks sufficient digital tools and technical support compared to several other well-studied sign languages. To reduce this gap, this study uses machine language learning (ML) techniques to convert KSL gestures into text and speech. The hand movements and facial expressions used in the KSL signs were gathered and examined for this purpose. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs), two examples of advanced machine-learning methods, have been used to create a reliable recognition system that can perform KSL in real time. The model has shown promising accuracy rates in the preliminary findings, instilling confidence in its potential to enhance KSL users' communication significantly. This study aims to improve education and daily communication among KSL speakers. The proposed model can be used at the regional Primary Healthcare Centers (PHC) for the effective interaction of a patient and doctors. In the future, we plan to expand the dataset, improve the algorithmic performance, and create user-friendly programs to make such programs useful at urban hospitals, too.