GhostPose: Multi-view Pose Estimation of Transparent Objects for Robot Hand Grasping

Pose estimation is a key challenge in robot manipulation and grasping task. Current object pose estimation approaches based on 3D models and depth sensor information have difficulties to handle transparent objects because of the limitation to capture the accurate depth information. To address these issues, we present a 6DoF pose estimation approach, called GhostPose, which utilizes a novel 3D bounding box prediction network and multi-view geometry with cameras on manipulator robot. Our 3D bounding box prediction network is simple and light-weight by adding a small branch to a one-stage object detector. The network detects 2D projections of 3D bounding box vertices. Then, 3D points are reconstructed from the 2D results of the multiple viewpoints with camera motion information, i.e. extrinsic parameters, calculated from the robot joint angles. We also present generalized pose definition to address pose ambiguity of symmetric objects and keep consistency of geometric properties around feature points across both of the symmetric and asymmetric objects. Comparing with the previous pose estimation approaches, GhostPose is more generalized to environments and object types, because it does not require 3D models, object specific key points, predefined stereo settings and depth map. In experiments, it outperforms a state-of-the-art approach and shows generalized properties by applying to a real manipulator robot grasping system.