Action Recognition Based on Correlated Codewords of Body Movements

Using spatio-temporal features is popular for action recognition. However, existing methods embed these local features into a global representation. Orders and correlations among local motions of each action are missing. This can make it difficult to distinguish closely related actions. This paper proposes a solution to address this challenge by encoding correlations of movements. Space-time interest points are detected in each action video. Then, feature descriptors are extracted from these key points and clustered into different codewords implicitly representing different characteristics of motions. The final representation of each action video is a combination of a bag of words and correlations between codewords. Then, the support vector machine is used as a classification tool. Based on the experimental results, the proposed method achieves a very promising performance and particularly outperforms the other existing methods that rely on spatio-temporal features.