Sign Language to Text Conversion Using Deep Learning Techniques | IEEE Conference Publication | IEEE Xplore

Sign Language to Text Conversion Using Deep Learning Techniques


Abstract:

The development of a sign language text translation system using deep learning and YOLO NAS (Neural Architecture Search) is an innovative application that bridges the com...Show More

Abstract:

The development of a sign language text translation system using deep learning and YOLO NAS (Neural Architecture Search) is an innovative application that bridges the communication gap. By integrating YOLO NAS for real-time sign language recognition, this model can accurately localize and identify sign language gestures. The deep learning component uses recurrent neural networks (RNNs) or transformer models to convert recognized characters into text. This innovation provides a comprehensive solution for the deaf and hard of hearing, allowing them to seamlessly communicate with the hearing world. Although this approach leverages the power of modern deep learning, existing methods have limitations. YOLO NAS is used to create a more efficient and accurate sign language translation system. Sign-to-Text powered by Mediapipeline is an innovative, research-based technology that makes communication easier for people who are hearing impaired. Combines advanced computer vision and natural language processing to convert sign language gestures into written or spoken words in real time. This research focuses on improving recognition accuracy and expanding vocabulary coverage, as well as improving accessibility and inclusiveness across different communication channels. We tested both models on American, Indian, and Japanese sign language datasets. Our model showed excellent accuracy of 97.8%, 96.67%, and 91.3% in American Sign Language, Indian Sign Language, and Japanese Sign Language, respectively.
Date of Conference: 08-10 February 2024
Date Added to IEEE Xplore: 04 April 2024
ISBN Information:
Conference Location: Indore, India

I. Introduction

The World Health Organisation states that, tens of millions of people globally—more than 5% of the worldwide population—have hearing loss, of which 34 million are teens (WHO). Studies indicate that by 2050, these figures might surpass 900 million. In addition, the majority of the time, a crippling hearing loss that impacts millions of people is most prevalent in low- and middle-incomenations. Those who are hard of hearing may communicate most naturally and expressively using sign language. Deaf isolation results from non-deaf persons never bothering to learn However, the discrepancy spoken language to communicate with the deaf community between deaf people and the broader public can reduced if computer is configured in such a way that language spoken can be translated into text. Those who are deaf or mute relyon sign language interpreter’s communication [1].

Contact IEEE to Subscribe

References

References is not available for this document.