A single camera, calibration free estimation and tracking scheme with multi-sensor fusion in a manufacturing workcell

The problem of tracking and grasping a moving part on a rotating turntable with the aid of a robotic manipulator is considered. The position and orientation of the part with respect to the turntable is assumed to be apriori unknown. Likewise the position and orientation of the robot with respect to the turntable is also assumed to be apriori unknown. In the algorithm described in this paper, the position and orientation information are estimated visually and recursively updated. The procedure involves "sensor fusion" with encoders placed on the turntable and encoders on the robot. Having obtained a good estimate of the position and orientation of the part with respect to the base frame of the robot, the manipulator is controlled to track and grasp the moving part. The operation of grasping has been demonstrated experimentally. The paper emphasizes various cases of the estimation scheme as a result of choosing various geometric cues on the end effector.