Reassessing Hierarchical Representation for Action Recognition in Still Images

Typical still action recognition methods rely on human body part detection and object detection. However, current human body part detectors and object detectors are far from perfect, leading to a negative impact on subsequent spatial relation learning of human-object interactions (HOIs). Bag-of-features (BoF)-based methods go beyond such modeling paradigms, but they do not achieve the state-of-the-art accuracies. In this paper, we propose two still action recognition methods that model HOI layouts by image hierarchical representation, rather than explicitly constructing HOI relations. The first method encodes a dense set of SIFT features using Fisher vectors, where an image is divided into increasingly fine regions with the spatial pyramid. The second method takes recent pretrained deep networks as feature execrators, where an image is divided into overlapped regions. The improvement effect of the hierarchical representation is proven by extensive comparison experiments. Our methods are very simple and easy-to-use, which remarkably outperform those BoF-based methods and complicated human-centric methods. To the best of our knowledge, our methods achieve the highest accuracies to date on the Sports, PPMI, and extended PPMI data sets.