Unsupervised Action Recognition Using Spatiotemporal, Adaptive, and Attention-Guided Refining-Network

Previous articles on unsupervised skeleton-based action recognition primarily focused on strategies for utilizing features to drive model optimization through methods like contrastive learning and reconstruction. However, designing application-level strategies poses challenges. This article shifts the focus to the generation-level modelings and introduces the spatiotemporal adaptively attentions-guided refining network (AgRNet). AgRNet approaches the reduction of costs and enhancement of efficiency by constructing the adaptive activity-guided attention (AAGA) and adaptive dominant-guided attenuation (ADGA) modules. The AAGA leverages the sparsity of the correlation matrix in the attention mechanism to adaptively filter and retain the active components of the sequence during the modeling process. The ADGA embeds the local dominant features of the sequence, obtained through convolutional distillation, into the globally dominant features under the attention mechanism, guided by the defined attenuation factor. Additionally, the progressive feature modeling (PFM) module is introduced to complement the progressive features in motion sequences that were overlooked by AAGA and ADGA. AgRNet shows efficiency on three public datasets, NTU-RGBD 60, NTU-RGBD 120, and UWA3D.