Signify: A Transformer-based Approach for Real-Time Speech to Sign Language Translation

Signify is a real-time system for translating voice to sign language, utilizing sophisticated transformer-based architectures, such as Wav2Vec 2.0 for speech recognition and Vision Transformers (ViT) for gesture representation. The system demonstrates superior performance on essential assessment criteria, with an accuracy of 96.5%, an F1-score of 96.1%, and a precision of 95.3%, surpassing traditional architectures like ConvLSTM and Transformer-TTS. The system eliminates dependence on manual transcription and enables comprehensive translation via a Flask-based web application, ensuring reliable performance in diverse acoustic environments and among different speaker profiles. Experimental assessments performed on benchmark datasets further validate the system's scalability, adaptability, and appropriateness for real-time implementation. The architecture addresses deficiencies in accessibility tools for the hearing-impaired community, providing a modular approach that allows for future enhancements, including dynamic gesture animation and language support.