An Optimization Model for Human Activity Recognition Inspired by Information on Human-Object Interaction

Human activity categories can be mainly divided into two types: body motion (e.g. "dancing" and "jumping") and human-object interaction (e.g. "playing guitar" and "riding bike"). In this paper, we propose a model that uses this insight to combine spatiotemporal features with information on human-object interaction to predict human activity categories in realistic videos. In this paper, we (1) improve on prior work by proposing a model to provide long-term prediction based on spatiotemporal features extracted by a deep 3-dimensional convolutional network and (2) extend the spatiotemporal features by combining with information on human-object interaction generated from an object detection model. Our model achieves 92.5% accuracy on UCF101 dataset that outperforms any other state-of-the-art methods. In addition, our approach has high computing efficiency and achieves real-time processing.