Visual Servoing of a 5-DOF Mobile Manipulator Using a Panoramic Vision System

In this paper, a novel visual servoing technique for a S-DOF mobile manipulator with an eye-to-hand camera configuration is introduced. The proposed technique can be categorized as an image based (or 2D) visual servoing using a fixed camera in conjunction with a conic mirror (aka, an omni-directional camera system) providing panoramic vision. Two fictitious landmarks mounted on robot's end-effector along with their mirror reflections, when viewed by the camera, provide enough information for 3D reasoning based on the four points viewed on the image plane. Instead of directly using the image features associated with these four points, five new image features are chosen to make the image Jacobian of full rank. A dual estimation/control strategy based on Extended Kalman Filter (EKF) is utilized to (I) estimate camera's intrinsic and extrinsic parameters, and (2) track the coordinates of the landmarks and their reflections on the image plane. The relationship between the translational and rotational velocity of a frame attached to the robot's end-effector and rate of change of the proposed image features are fully formulated The robustness of the proposed visual servoing technique is illustrated through computer simulations.