Grasping Objects Sequentially Using Expanded Segmentation Masks and Linear Combination of RGB-D Heightmaps

The main challenge tackled in this paper is to grasp objects specified by the user sequentially in a cluttered scene. We propose a target-driven Sequential Grasping Network (SGN) to grasp objects in the bin-picking context, which is mainly composed of a Mask R-CNN for instance segmentation and a modified Visual-Pushing-Grasping (VPG) network for grasping evaluation. For the segmentation network, we propose a neighborhood expansion method of the mask based on the size of the target object to facilitate grasps. For the grasping network, we present an approach using the linear combination of the RGB-D heightmaps as the input, this work is to reduce the size of grasping policy network. For the sequential grasping tasks, we have quantitatively evaluated the performance of the proposed SGN on the dataset. The average completion rate of our SGN is 93.3% in the separated scenes and 62.5% in the occluded scenes. The results show that our approach achieves a remarkable performance in the sequential grasping tasks.