Indian Sign Language Recognition using Skin Segmentation and Vision Transformer

Sign Language is the common mode of communication among the speech and hearing-impaired people, but interpreting this language becomes a challenge for others who don’t practise it. To bridge this communication gap, many Artificial Intelligence based models have been designed worldwide. In the Indian context, the field is still relatively new. A 72-word self-created Indian Sign Language dataset has been used in this study. A Vision transformer model, consisting of just 2 transformer layers has been proposed. The pre-processing used on the images are YCbCr conversion and morphological operation based skin segmentation. The model achieves a test accuracy of 99.56% and experimentation with different publicly-available datasets confirms its superiority over previous methods.