Collision-free visual servoing of an eye-in-hand manipulator via constraint-aware planning and control

A constraint-aware eye-in-hand visual servoing control law is proposed. The control law is designed for a robot manipulator with an uncalibrated camera mounted on its end-effector. The method uses an image-based command to describe the desired end-effector position with respect to an object with an unknown position. When starting from an unknown position, the control law uses feedback from the camera to move the robot towards the reference image while satisfying a set of system constraints. The visual servoing control law is implemented via a nonlinear model predictive control framework to generate feasible and realistic robot trajectories that respect the robot's joint limits and velocity limits. The control law explicitly keeps the target object within the camera's field of view and avoids potential collisions with workspace obstacles. An appropriate representation of the robot's whole-arm collision constraints is extracted from well-known path planning methods, such as probabilistic road maps and dynamic collision checking algorithms. Experiments using an uncalibrated eye in-hand platform demonstrate the ability of the visual servoing control law to achieve closed-loop positioning via collision-free trajectories, even when the initial object location is uncertain.