Small Human Group Detection and Event Representation Based on Cognitive Semantics

To recognize concurrent human activities in videos, we proposed a cognitive semantics based novel event representation for small human group detection and event recognition. Given a video with human detection and tracking results, the video is firstly described by cognitive linguistic primitives, including "paths", "places", "things", "actions", and "causes". Then the structural and semantic distance of "things" (i.e. human individuals) in the same "place" will be calculated, and all the similar "things" will be merged together to reduce the semantic social entropy with regard to the entire "path". Once a group of "things" (i.e. a human group) is identified, its "actions" will be classified into atom group activities by their corresponding spatial and temporal semantics. The spatial and temporal similarity of atom group activities is examined and a probabilistic context free grammar is derived from these atom activities based on Minimum Description Length (MDL) criterion. The induced grammar rules will then be used to parse test videos represented by cognitive linguistic primitives. The proposed novel video event representation can be used to describe and recognize complex human activities, including both individual and group actions. The experimental results on the BEHAVE and Collective datasets have demonstrated the effectiveness of the proposed method.