Cascaded Time Delay Compensation and Sensor Data Fusion for Visual Servoing

The main disadvantage of vision-based control systems is the inherent time delay due to the processing of the image information, which leads to significant performance losses of the control system. We present a cascaded Kalman Filter structure to compensate for the time delay introduced by the 6d pose estimation in vision based control systems. In addition to delay compensation the structure allows all sensor data to be used asynchronously for state correction. The first Kalman Filter runs in the past and compensates for the time delay of the pose estimation. The second Kalman Filter is used exclusively for sensor data fusion. A special structure within the filter allows all sensor data to be used for state correction regardless of their sampling rate. The sampling rate, at which the second Kalman filter provides the state vector for a controller, can be set independently of all other variables.The entire structure has been tested by simulation and in experiments on a three degree of freedom robot. By applying the cascaded Kalman Filter structure, the position as well as angular errors in orientation can be reduced by more than 50%.