Eye-hand online adaptation during reaching tasks in a humanoid robot

In this paper we propose a method for the online adaptation of a humanoid robot's arm kinematics, using its visual and proprioceptive sensors. A typical reaching movement starts with a ballistic open-loop phase to bring the hand to the vicinity of the object. During this phase, as soon as the hand of the robot enters the field of view of one of its cameras, a vision based 3D hand pose estimation method feeds a particle filter that gradually adjusts the arm kinematics' parameters. Our method makes use of a 3D CAD model of the robot hand (geometry and texture) whose predicted position in the image is compared at each time step with the cameras' incoming information. When the hand gets close to the object, the kinematic errors have reduced significantly and a better control of grasping can eventually be achieved. We have tested the method both in simulation and with the real robot and verify error decreases by a factor of 3 during a typical reaching time span.