Manipulator control by using servoing with the stereo vision

A new method of visual servoing with the stereo vision to control the position and orientation of the manipulator with respect to an object is presented. Conventional control methods by visual servoing use a monocular camera and have several problems. For example, either shape information or desired distance of the target object from the camera must be given. Furthermore, a stability problem arises unless the initial positional error of features in the image is sufficiently small. These problems are caused by the image Jacobian matrix, that is, its provisional value at a desired position is used instead of the correct one. By using stereo vision, the image Jacobian matrix can be calculated correctly at any position, so neither shape information nor desired distance of the target object is required. Also, stability is assured even if the initial error is not small. Both simulation and experimental results are presented to demonstrate the effectiveness of this method.