Event recognition based-on social roles in continuous video

In this paper, we present a new method for video event recognition based on social roles of agents, which are inferred from their daily activities in continuous video. This is motivated from the observation that people have their social roles, and the information of social roles in certain scene provides useful cues for recognizing video events. First, events are represented by an And-Or Graph (AOG), which can represent both the hierarchical decompositions from events, sub-events and atomic actions and the contexts for temporal relations. Then, a model of social roles is proposed to infer the roles of the agents in continuous video. Finally, an improved event parsing algorithm based on social roles context is adopted to recognize events. Experimental results show that our method is effective in performing inference tasks of social roles and can improve performance of event recognition.