Skeleton-based action recognition using spatio-temporal features with convolutional neural networks

Deep learning methods achieved outstanding performance in 3D action recognition. Due to time-series property of skeleton sequences, skeleton based action recognition methods are mostly based on recurrent neural networks and Long Short-Term Memory networks. Compared with RRN-based methods, in this paper a new method was proposed to encode two types of spatial features and motion between the 3D coordinates of joints in two frame with eight time-step gap into a color image. Encoding skeleton sequences into an image made it possible to train a convolutional neural network on each type of images. The proposed method achieved state-of-the-art performance on NTU RGB+D action recognition dataset. The results show an accuracy of 84.9% for action classification on this dataset.