Vision Transformer-enabled Semantic Segmentation for Enhanced Robotic Grasping in 3D Space

In recent years, the integration of computer vision and robotics has gathered significant interest in automating a range of tasks, including object detection and robotic grasping in 3D space. This research paper introduces an innovative approach to enhance robotic grasping in 3D space by leveraging unsupervised semantic segmentation using a Vision Transformer (ViT) based architecture. The proposed method utilizes a deep learning architecture to segment objects in the scene and incorporates a grasp detection module to identify potential grasping rectangles based on the segmented objects. The model is trained on a large-scale dataset, enabling it to generalize effectively across various objects and environments. The paper also addresses the utilization of 2D bounding box and point cloud data to facilitate 3D object localization. The proposed approach is evaluated using the Cornell Grasping Dataset, achieving a remarkable success rate of 87.7% for grasping rectangles. Furthermore, by leveraging the 2D bounding box generated by the model alongside corresponding point cloud data, the method estimates the object’s height and orientation, leading to the generation of a comprehensive 3D bounding box. This paper presents a robust and efficient method for robotic grasping and 3D object localization by harnessing the capabilities of computer vision and deep learning techniques.