Deep Learning Technology to Recognize American Sign Language Alphabet Using Mulit-Focus Image Fusion Technique

In our study, we implemented an innovative approach that harnessed the proven capabilities of ResNet, which achieved an exceptional accuracy rate of 99.98 % in classifying American Sign Language (ASL) alphabets. This approach involved utilizing ResNet to extract features from multiple input images, subsequently amalgamating these features and feeding them into the Vision Transformer (ViT) model for further processing. The underlying strategy of knowledge transfer exemplifies the principles of transfer learning and multi-image-focused fusion. As a result of our efforts, we observed a significant enhancement in the ViT model's accuracy, which surged from 88.59% to an impressive 97.09 %. This accomplishment underscores the potential of our proposed system in advancing ASL alphabet classification tasks, with particular relevance to individuals with hearing impairments. Our study not only emphasizes the significance of innovative fusion techniques in deep learning but also presents a promising solution for enhancing the accuracy and reliability of ViT in image processing applications. By exploring the intersections of various neural network architectures, our work paves the way for more inclusive and effective technologies, fostering seamless engagement for individuals with diverse abilities.