Close Human Interaction Recognition Using Patch-Aware Models

This paper addresses the problem of recognizing human interactions with close physical contact from videos. Due to ambiguities in feature-to-person assignments and frequent occlusions in close interactions, it is difficult to accurately extract the interacting people. This degrades the recognition performance. We, therefore, propose a hierarchical model, which recognizes close interactions and infers supporting regions for each interacting individual simultaneously. Our model associates a set of hidden variables with spatiotemporal patches and discriminatively infers their states, which indicate the person that the patches belong to. This patch-aware representation explicitly models and accounts for discriminative supporting regions for individuals, and thus overcomes the problem of ambiguities in feature assignments. Moreover, we incorporate the prior for the patches to deal with frequent occlusions during interactions. Using the discriminative supporting regions, our model builds cleaner features for individual action recognition and interaction recognition. Extensive experiments are performed on the BIT-Interaction data set and the UT-Interaction data set set #1 and set #2, and validate the effectiveness of our approach.