Visual servo control for coordinated manipulation of a mobile two-arm robot

Two PZT cameras are used in this paper to realize visual servo control for coordinated manipulation of a mobile two-arm robot. The focus of this study is mainly on the problems of detection, recognition and localization of a target object in an indoor environment of scattered background and varying illumination. First, a modified algorithm is proposed in the HSV color space to detect objects based on updated segmentation thresholds and bounding rectangles, thereby improving its adaptability to illumination variation. Second, the target is identified from the detected objects using the target contour features and the Hu's moment invariants. Then, the spatial coordinates of the target object are obtained with the cameras' perspective matrices, and the target position is determined. Finally, to verify effectiveness of the image processing algorithms presented and the visual servo control, experiments are done on a robot platform to mimic the manipulation of pouring water with two arms, and the results show that the robot system can complete the task successfully.