Real-time cooperative behavior acquisition by a humanoid apprentice

An apprentice is an able-bodied individual that should interactively assist an expert, and through this interaction, they should acquire knowledge and skill in the given task domain. In this context the robot should have a useful repertoire of sensory-motor acts that the human can command with spoken language. In order to address the additional requirements for learning new behaviors, the robot should additionally have a real-time behavioral sequence acquisition capability. The learned sequences should function as executable procedures that can operate in a flexible manner that are not rigidly sensitive to initial conditions. The current research develops these capabilities in a real-time control system for the HRP-2 humanoid. The task domain involves a human and the HRP-2 working together to assemble a piece of furniture. We previously defined a system for Spoken Language Programming (SLP) that allowed the user to guide the robot through an arbitrary, task relevant, motor sequence via spoken commands, and to store this sequence as re-usable macro. The current research significantly extends the SPL system: It integrates vision and motion planning into the SLP framework, providing a new level of flexibility in the behviora that can be created. Most important it allows the user to create ldquogenericrdquo functions with arguments (e.g. Give me X), and it allows multiple functions to be created. We thus demonstrate - for the first time - a humanoid robot equipped with vision based grasping, and the ability to acquire multiple sensory motor behavioral procedures in real-time through SLP in the context of a cooperative task. The humanoid robot thus acquires new sensory motor skills that significantly facilitate the cooperative human-robot interaction.