Combining force and visual feedback for physical interaction tasks in humanoid robots

In this paper we present a framework for combining force and visual feedback in the task space to deal with humanoid interaction tasks like open doors in a real kitchen environment. We present stereo-vision methods for markerless recognition and estimation of environmetal elements for applying force control strategies for compliant execution using a 6D force-torque sensor mounted in the humanoids's wrist. The framework consists of components for model-based self-localization, visual planning, door recognition, grasping based on visual servoing, real time handle tracking and task execution considering force feedback during the physical interaction with the environment. Experimental results on the humanoid robot ARMAR-IIIa are presented.