Visual tracking for moving multiple objects: an integration of vision and control

This paper addresses the use of a vision sensor in the feedback loop for tracking an object which is selected from multiple objects that are unknown and randomly moving on a conveyor belt. For achieving such a tracking task, a position-based visual tracking system is proposed. This system has a visual processing part in which multiple objects are segmented. Here, the authors address the segmentation problem as a clustering problem. In its control part, PD and self-tuning controllers are designed and compared with respect to their tracking speed and noise immunity performances. As the tracking device, a pan-tilt camera mount is proposed initially and the performance of the proposed visual servoing system is evaluated by simulations. The real-time applicability of the introduced vision system is then experimented on a Manutec R15 robotic manipulator in picking parts on a conveyor belt. The results show that the introduced system can be used as a real-time system.