VRFNet-ASLiT: Fused Deep CNN and Adaptive Super Resolution Transform-Based Hand Gesture Recognition

Hand gesture recognition plays a vital role in human-computer interaction, offering a natural and intuitive means of communication. However, vision-based hand gesture recognition is sensitive to variations in lighting conditions and may face challenges in accurately distinguishing between similar gestures or subtle variations in hand. To address this concern, a hybrid deep convolutional neural network (CNN) model, which is the fusion of VGG19 and ResNet50 (VRFNet), is proposed to encapsulate the intricacies of subtle variations or infrequent gestures. The proposed method incorporates the adaptive super resolution transform (ASLiT) technique to generate the high-resolution time-frequency representation of input images, and VRFNet effectively classifies the different hand gestures. The proposed method is tested on LeapGestRecog dataset, Modified National Institute of Standards and Technology (MNIST) digits dataset, and American Sign Language (ASL) digits and provides a recognition accuracy of 99.97%, 99.6%, and 98.8%, respectively. In addition to the proposed VRFNet model, the comparison of the performance with other two other models: AlexNet and VGG19, is presented. The results reveal that VRFNet model exhibits exceptional performance in hand gesture recognition tasks with improved a precision of 99.83%, 99.64%, and 99.05%, a recall of 99.86%, 99.62%, and 99.85%, and an F1 score of 98.85%, 99.63%, and 98.96%, respectively, across the LeapGestRecog, MNIST, and ASL digits dataset. These results underscore the model’s remarkable accuracy in recognizing hand gestures within diverse datasets. The proposed VRFNet model is also competitively, comparable to other state-of-the-art hand gesture recognition techniques. Furthermore, an ablation study is conducted to analyze the impact of individual models for hand gesture recognition.