Temporal Dynamics and Self-Attention: Adapting Vision Transformers for Sequential Sign Language Data

Sign Language Translation (SLT) has major challenges in capturing complex spatiotemporal relationships and achieving real-time performance because of the limits of traditional CNN-based methods. The study addresses these limitations by processing sequential data of sign language with Vision Transformers (ViTs). It deals particularly with time and self-attention. The proposed system unites Sign2Text and SLRFormer architecture to make end-to-end translation that at the same time does not require the involvement of an intermediate gloss annotation. According to the experimental results, ViT-based models are equal to 46.8 BLEU, 18.5 percent of Word Error Rate, and 83.2 percent measure of accuracy with inference of $\mathbf{4 5 ~ m s}$ between any single frame. Error rates using these results are $\mathbf{1 1. 6}$ points better in the BLEU score and have a lower Word Error Rate by $10.2 \%$ as compared to CNN-RNN hybrid approaches. This system successfully deals with multimodal representations such as gestures, facial expression and body pose in providing solutions to accessibility of the Deaf and Hard of Hearing population.