Learning Visuomotor Transformations and End Effector Appearance by Local Visual Consistency

We present an algorithm that enables a robot to learn the visuomotor transformation from its joint angle space to visual space. The learned transformation can accurately predict location and shape of robot end effector's image projection. This paper extends past work by approximating the end effector by a planar region, rather than a point, in 3-D space, and through its use of spatially and temporally local, rather than global, measures of image consistency. Our robotic experiments demonstrate that the proposed algorithm can learn location and shape of the image region corresponding to the end effector, and how it deforms as the arm moves randomly in front of the camera. Our approach does not require that the end effector be identified with a specific marker. We also demonstrate that the region corresponding to the end effector can adapt to changes in the end effector shape.