Enhancing One-Shot Action Recognition from Occluded Skeletons with a Spatial-Temporal Denoising AutoEncoder

In this paper, we tackle the challenge of one-shot action recognition from occluded skeleton data. The task uses limited labeled samples to learn an action recognition model to cope with the time-consuming and laborious problem of annotating a large amount of data. Additionally, the unavoidable occlusion of human joints in real-life scenarios significantly complicates skeleton action recognition efforts. To address these issues, we introduce a novel Spatial-Temporal Denoising AutoEncoder (ST-DAE) designed for skeleton-based one-shot action recognition. Employing an encoder-decoder architecture, the encoder extracts intrinsic patterns of movements from input occluded skeletons, whereas the decoder learns to reconstruct the skeletons under the supervision of clean skeletons. Then a Spatial-Temporal GCN (ST-GCN) backbone network learns to map the reconstructed skeletons into a high-dimensional feature space to capture their essential characteristics for action recognition. Finally, one-shot action recognition is achieved by a KNN classifier in the learned feature space. Experimental validations on the comprehensive NTU-RGB+D 120 RE occlusion dataset demonstrate the effectiveness of the proposed spatial-temporal denoising autoencoder against noisy skeleton data.