Challenges of Vision for Real-Time Sensor Based Control

Sensor-based robot control allows manipulation in dynamic environments with uncertainties. Vision offers a low-cost sensor modality, but low sample rate, high sensor delay and uncertain measurements limit its usability. This paper addresses three problems: uncertain visual measurements, different sampling rates and compensation of the sensor delay. To alleviate the problems above an approach for visual tracking of a moving object with end-effector mounted camera is presented. The pose of the object relative to the camera is determined with model based pose estimation method. The absolute pose and velocity of the target object are estimated by fusing visual measurements over time. A low sample rate visual measurement with sensor delay is integrated with the pose of the end-effector in an extended Kalman filter. Experiments with a 5-DOF parallel hydraulic manipulator show that integration of several measurements together with sensor delay compensation significantly reduce oscillations and phase shift in visual control.