Structured light camera base 3D visual perception and tracking application system with robot grasping task

3D vision-based recognition as well as grasping of complex objects is required not only for detection and categorization but also for pose estimation and robotic pick-and-place operations. In this paper, we propose a structured light camera based 3D visual perception and tracking system application with robot arm grasping for manufacturing. In the first step, we use a geometric surface primitive patch segmentation approach based on Hough transforms to obtain accurate surface normal estimations from 3D point clouds for the identification of patch primitives. The most relevant primitives for our application include planar and cylindrical surface patches. We extract primitive surface patches from automotive CAD models in DXF or 3DS format. The models are then decomposed to simple entities such as planar polygons, vertexes and lines. Our system takes advantage of the available CAD data for both object recognition and for pose estimation. In the final step, we propose point based KLT tracking method with dominant features from object recognition results both raw data and CAD. Our experimental results demonstrate that we can show application results from YASKAWA MOTOMAN robots demonstration. Whole processes perform within few seconds, an accurate pose and reasonable tracking.