Spatial–Temporal Transformer for Optimizing Human Health Through Skeleton-Based Body Sports Action Recognition

Accurate recognition of human actions and poses is essential for applications in fitness monitoring, rehabilitation, and virtual coaching. Despite progress in skeleton-based recognition using Graph Convolutional Networks (GCNs) and Transformers, existing methods often fail to effectively model complex spatial-temporal dependencies, especially in dynamic or subtle actions. To address these gaps, we propose the Focal and Global Spatial-Temporal Transformer (FG-STTrans) framework, a novel multi-stage architecture designed to capture both localized and global spatial-temporal dependencies with high precision. The FG-STTrans framework incorporates key innovations, including Basic Spatial Transformer (BSTrans) and Basic Temporal Transformer (BTTrans) blocks for early-stage feature extraction, and advanced modules such as Joint-Region Cross-Attention (JRCA) for integrating focal joint and global region-level features. Additionally, the Focal and Global Temporal Transformer (FG-TTrans) module leverages dilated temporal convolutions and self-attention mechanisms to model both short-term and long-term temporal relationships. These components collectively enable the framework to handle complex co-movement patterns, occlusions, and variability in execution styles. We evaluate FG-STTrans on two diverse datasets: the Workout Action Video Dataset (WAVd) and the YogaVid dataset. The proposed framework achieves state-of-the-art performance, with an accuracy of 94.25% on WAVd and 90.70% on YogaVid datasets. These results demonstrate the robustness and generalizability of FG-STTrans in recognizing a wide range of workout actions and yoga poses.