Guiding attention for grasping tasks by gestural instruction: the GRAVIS-robot architecture

A major goal for the realization of a new generation of intelligent robots is the capability of instructing work tasks by interactive demonstration. To make such a process efficient and convenient for the human user requires that both the robot and the user can establish and maintain a common focus of attention. We describe a hybrid architecture that combines neural networks and finite stale machines into a flexible framework for controlling the behaviour of a vision based robot called GRAVIS-robot (Gestural Recognition Active Vision System robot). It consists of a binocular camera head, a 6 DOF robot arm and a 9 DOF multifingered hand. We focus primarily on nonverbal communication based on gestural commands of a human instructor which will at a later stage be complemented by spoken instructions.