Understanding Human Behaviors with an Object Functional Role Perspective for Robotics

Intelligent robotic assistance requires a robot to accurately understand human behavior. Many researchers have explored human-object interactions to decode behavior-related information. However, current methods only model probabilistic correlations between objects and activities. Their applications are usually limited to fixed environments and fixed sets of activities. They are unable to deal with variability in the real environments due to the lack of the human-like cognitive reasoning process. To address this urgent problem, we developed an Object Functional Role Perspective method to endow a robot with comprehensive behavior understanding. Instead of using specific objects to identify an activity, our role-based method models the human cognitive process during task performing by analyzing object selection and object interaction. Then activity-related information, such as activity feasibility, likely plan, and urgent need of an activity, is inferred in order to improve a robot's cognition level for comprehensive behavior understanding. Through a large amount of human behavior observations, this cognitive knowledge is constructed using a Markov random field (MRF) model. Experiments were performed in both real-life scenarios and lab scenarios to evaluate the method's usefulness. The results demonstrated flexibility and effectiveness of the role-based method for human behavior understanding under variability.