An affordance-based pilot interface for high-level control of humanoid robots in supervised autonomy

In this work we present the concept of a pilot interface to control a humanoid robot on an abstract level in unknown environments. The environment is perceived with a stereo camera system and then simplified into a set of environmental primitives. Based on these primitives the interface proposes affordances to the pilot. Affordances are represented as certainty functions over the space of end-effector poses. The pilot operates the robot by selecting among proposed affordances and related action primitives, i.e. Object-Action Complexes (OACs). Before initiating execution, the pilot can review and revise the parameterization of the scheduled action primitive in a 3D reconstruction of the environment. The pilot interface proposed in this work has been implemented and evaluated on the humanoid robot WALK-MAN. With this work we also demonstrate the transferability of the perceptual concept, as our previous experiments have been performed using the humanoid robot ARMAR-III.