Learning of 6D Object Poses with Multi-task Point-wise Regression Deep Networks

6D pose estimation of known objects has received much attention for its wide range of applications, especially in robotic grasping. In recent deep learning methods, the 6D pose estimation problem can be converted into a translation-and-rotation regression problem. Here we propose a novel multi-task point-wise regression network for 6D pose estimation and a robotic grasping system equipped with the object 6D pose and a grasp detector. To reduce the gap between the estimated and real poses, a synthetic dataset for the 6D pose estimation network is generated in a physical engine using domain randomization. This network directly uses a point cloud with an XYZRGB formatted input. During the network training, the rotation regression is evaluated by a continuous 6D rotation representation. The proposed 6D pose estimation network outperforms high-performance networks in real-world experiments. To evaluate 6D pose estimation in the real world with uncertainty introduced by sensor noise, we propose a multi-camera vision system that fuses the sensor data from three RealSense D415 cameras and develop a grasp-detection algorithm based on the object 6D pose. The proposed 6D pose estimation-based robotic grasping framework performs precise object grasping in both simulated and real robot experiments.