Recognizing Micro Actions in Videos: Learning Motion Details via Segment-Level Temporal Pyramid

Recognizing micro actions from videos is a very challenging problem since they involve only subtle motions of body parts. In this paper, we propose a new deep-learning based method for micro action recognition by building a segment-level temporal pyramid to better capture the motion details. More specifically, we first temporally sample the input video for short segments and for each of the video segment, we employ a two-stream convolutional neural networks (CNNs) followed by a temporal pyramid for extracting deep features. Finally, the features derived from all the video segments are combined for action classification. We evaluate the proposed method on a micro-action video dataset, as well as a general-action video dataset, with very promising results.