Enhancing SAM-6D’s Object Pose Estimation with Depth-guided Segmentation

Accurate 6D object pose estimation is crucial for robotic interaction, enabling tasks like grasping and manipulation, particularly of novel objects in unseen environments. While zero-shot methods like SAM-6D, which leverages the Segment Anything Model (SAM), offer promise, they still face challenges, especially in cluttered scenes and under occlusions. SAM-6D’s segmentation, guided solely by RGB images, often suffers from over-segmentation due to its reliance on texture information. This research addresses this limitation by proposing a novel depth-guided segmentation method to enhance SAM-6D’s performance. We introduce a Merging Intersection-Based Method that leverages depth information to refine instance segmentation masks, processing depth images, evaluating segment intersections, and merging over-segmented regions. This allows SAM to incorporate crucial geometric cues, moving beyond its reliance on texture. Experimental results on the LineMOD benchmark dataset demonstrate a 3.35% improvement in segmentation IoU and reduces pose estimation error by 3.40%, contributing to more reliable 6D pose estimation and ultimately improving robot interaction with novel objects in complex environments.