A New Human Action Recognition Method Based on Skeletons From Depth Sensors

Skeleton data from depth sensors, enriched with substantial information and compact in storage, provides a streamlined sequential representation. It is resilient to environmental variables such as background noise, attire, and illumination. Consequently, skeleton-based human action recognition emerges as an important topic in computer vision. Its key task is to extract distinctive features, specifically those critical for differentiating human actions. This work proposes an ambiguous-feature-refinement (AFR) and orthogonal-basis-coordinate-transformation-(OBT)-based residual graph convolution neural network (AOnet) model for skeleton-based human action recognition. First, multiple input branches (MIBs) are proposed and employed to enrich skeletal information features at the early data-input stage while maintaining a compact and effective representation of data. Second, since many actions exhibit highly similar joint features, an AFR model is introduced to further distinguish ambiguous features without considering external information interaction. This step benefits the extraction of distinctive spatial-temporal (ST) features, thereby improving the differentiation among action categories. Finally, an OBT method is proposed to convert refined nonlinear ST features into linear ones and augment the model’s ability to capture higher order dynamic information. The proposed method has been evaluated on two large-scale datasets. The experimental results confirm that AOnet demonstrates superior performance over the state of the art.