Robust Image Processing and PositionBased Visual Servoing

This chapter presents image processing techniques and issues that are related to position-based robot visual servo control. It examines the trade-offs between the requirements of speed, accuracy and robustness. To achieve the appropriate balance of these requirements, directed image processing is implemented which uses windowing techniques, prediction techniques, and a priori information regarding the features of the target object. Details of the binary image processing used to measure hole and corner feature locations on the image plane are presented. Kalman filtering is used to estimate the relative pose of the target object with respect to the camera, and this information in conjunction with the object description, is used to predict the window size and location for each feature on the next sample. In addition, feature planning techniques are used to provide a database which forms the basis for real-time feature switching along the relative motion trajectory to maintain a suitable set of visible features for visual servoing. Finally, sensor fusion techniques, based on the Kalman filter, are presented to show how measurements from multiple cameras, or a camera and range sensor can be properly integrated to improve the robustness and accuracy of the pose estimation, based on the image processing results.