Spatiotemporal View-Reset Deep Learning With Attentional GRUs for Skeleton-Based Human Action Recognition

Skeleton-based human action recognition has attracted significant attention. However, the skeleton spatial invariance and temporal context modeling are recent challenges for most existing methods. This work proposes a view-reset network, which integrates two important branches: spatial view-reset module (SVRM) and temporal attention module (TAM). In the SVRM, the skeleton model from different perspectives is reset in a unified coordinate system, eliminating the influence of viewpoint changes. In the TAM, the perception of temporal features is jointly enhanced by weighting each frame’s importance in the context. Furthermore, the pretrained residual network (ResNet) is used for prediction. The sample size is increased through data augmentation to improve the robustness of the model. The SVRM, TAM, and ResNet form an end-to-end learning network. The ablation study proved that the model could record the key skeletons and frames in the sequence and then reset the human body to a new position, making it easy for learning. The proposed model is evaluated on four challenging benchmarks based on the performance of the cross-view evaluation metrics. Experiments prove that the proposed model has superior performance and surpasses many state-of-the-art algorithms, with an increase of 1.91% over the top ten on the NTU RGB+D 60 dataset.