Task-model based human robot cooperation using vision

In order to assist a human, the robot must recognize human motion in real time by vision, and must plan and execute the needed assistance motion based on the task purpose and the context. In this research, we tried to solve such problems. We defined the abstract task model, analyzed the human demonstration by using events and an event stack, and automatically generated the task models needed in the assistance by the robot. The robot planned and executed the appropriate assistance motions based on the task: models according to the human motions in the cooperation with the human. We implemented a 3D object recognition system and a human grasp recognition system by using trinocular stereo color cameras and a real time range finder. The effectiveness of these methods was tested through an experiment in which the human and the robotic hand assembled toy parts in cooperation.