Organization of behavioral knowledge from extraction of temporal-spatial features of human whole body motions

This paper describes an approach to structuring behavioral knowledge based on classification of human whole body motions and extraction of the behavioral transitions. The motion patterns are learned by Hidden Markov Models (HMMs), which can be used for classification of the motion patterns. The HMMs are called “motion symbol” since They abstract their corresponding motion patterns. The motion patterns are organized into a hierarchical tree structure (“motion symbol tree”) representing the property of similarity among the motion patterns. The motion patterns are classified based on the motion symbol tree. Concatenated sequences of motion patterns are stochastically represented as transitions between the abstracted motion patterns by using an N-gram Model (“motion symbol graph”), and the transitional relationships of the human behaviors are extracted. The integration of the motion symbol tree and the motion symbol graph makes it possible to recognize motion patterns fast and predict human behavior during observation. The experiments on a large motion dataset validate the proposed framework.