Deep learning for graph analysis: application to online human activity recognition

Human Activity Recognition (HAR) using sensor data is a time series classification problem. The challenge lies in effectively managing spatial and temporal dependencies while highlighting crucial data variations. This can be accomplished through 3D skeleton data from an RGB+D camera. The hypothesis of this work is that representing 3D skeleton data as spatio-temporal graphs and utilizing graph embedding techniques and deep learning model, composed from Graph Convolutional Networks (GCN), to learn relationships between joints, and Transformers to focus attention on relevant parts, will improve the performance of HAR. We opted to design this model utilizing a Hill Climbing approach. To facilitate real-time action recognition, we employed the sliding window approach. This method involves segmenting the continuous data stream of 3D skeletons, potentially sourced from a Kinect device, into sequences of uniform length. The article shows a significant improvement of the accuracy of the learning according to the state of the art.