Improved rank pooling strategy for complex action recognition

Feature ranking from video-wide temporal evolution brings reliable information for complex action recognition. However, a video may contain similar features in the sequence of frames which deliver unnecessary information to the ranking function. This paper proposes a method to improve the rank-pooling strategy which captures the optimized latent structure of the video sequence data. The optimization is followed by removing the redundant features from the sequence data. The cosine and correlation distance metrics are employed to detect the identical features and extract the most efficient information from the video frames. Then, the ranked features are generated from the optimized and clean sequence data. The proposed improvement is easy to implement, fast to compute and effective in recognizing complex actions. As a result, the proposed approach reaches remarkable action recognition performance on benchmark datasets, namely Hollywood2, URADL, and HMDB51. The results are further compared with state-of-the-art techniques in the experiment section to confirm the effectiveness of the improved rank pooling framework.