Skeleton-Based Self-Supervised Feature Extraction for Improved Dynamic Hand Gesture Recognition

Human-computer interaction (HCI) has become integral to modern life, especially in digital environments. However, challenges persist in utilizing hand gestures due to factors such as the dynamic nature of gestures and the intricacies of intra and inter-finger movements. In this paper, we propose an innovative approach to improve skeleton-based hand gesture recognition by integrating self-supervised learning, a promising technique for acquiring distinctive representations directly from unlabeled data. The proposed method takes advantage of prior knowledge of hand topology, combining topology-aware self-supervised learning with a customized skeleton-based architecture to derive meaningful representations from skeleton data under different hand poses. We introduce customized masking strategies for skeletal hand data and design a model architecture that incorporates spatial connectivity information, improving the model's understanding of the interrelationships between hand joints. The extensive experiments demonstrate the effectiveness of the approach, with state-of-the-art performance on benchmark datasets. An exploration of the generalization of learned representations across datasets and a study of the impact of fine-tuning with limited labeled data are conducted, highlighting the adaptability and robustness of our approach.