SkeletonMix: A Mixup-Based Data Augmentation Framework for Skeleton-Based Action Recognition

Skeleton-based human action recognition has received widespread attention for its robustness to changes in the background and appearance of actors compared to the RGB modality. Data augmentation is widely used to explicitly regularize the model to prevent overfitting, especially when the number of labeled samples is scarce. However, compared to various augmentation methods available for the RGB modality, there are fewer works on the skeleton modality, especially a model-agnostic augmentation method that can be easily integrated into multiple models. We address the problem by proposing a comprehensive data augmentation framework named SkeletonMix, which contains a pair sample selection module for mixup and random augmentations tailored for skeleton modality. SkeletonMix is a non-learning framework and can be applied to different models in a plug-and-play manner. Extensive experiments on NTU RGB+D, NTU RGB+D 120 and PKU-MMD datasets demonstrate the effectiveness of our proposed framework under limited labeled training data. Our proposed method boosts the performance by a maximum of 7.5% on scarce training data setup (5% of training data).