A hybrid neural network based vision-guided robotic system

There are two primary methods for mapping an input image to robot motion: computed kinematics and visual servoing. Computed kinematics uses a kinematic transform between the image plane and the world frame. Computed kinematics algorithms require only a single iteration, but are sensitive to calibration errors. Visual servoing uses a control law to regulate the image to a desired state. Visual servoing is more robust, but requires more computation to reach a solution. To balance these opposing factors, we proposed a hybrid system that uses an initial computed kinematics move followed by a visual servoing correction, thereby providing a compromise between speed and accuracy. A linear approximation model and a neural network were used to approximate the kinematic transform between the image and world frames. A PD control system is used to regulate the image to its final state.