Detecting Object Surface Keypoints From a Single RGB Image via Deep Learning Network for 6-DoF Pose Estimation

Estimating the 6-DoF (Degree of Freedom) object pose from a single RGB image is one of the challenging tasks in the field of computer vision. Before the pose which is defined as the translation and rotation parameters can be derived by the traditional PnP algorithm, 2D image projections of a set of 3D object keypoints must be accurately detected. In this paper, we present techniques for defining 3D object surface keypoints and predicting their corresponding 2D counterparts via deep-learning network architectures. The main technique to designate 3D object keypoints is to employ quadratic fitting scheme for calculating the principal surface curvatures as the weights and then select from all surface points the ones mostly distributive with larger curvatures to describe the object shape as possible. However, the 2D projected keypoints are not directly regressed from the network, but encoded as the unit vector fields pointing to them, so that the voting scheme to recover back those 2D keypoints can be performed. Moreover, an effective loss function with the regularization term is adopted in training ResNet for predicting image projections of object keypoints by focusing on small-scale errors. Experimental results show that our proposed technique outperforms state-of-the-art approaches in both “2D projection” and “3D transformation” metrics.