Improvement of Zero-shot Object Segmentation and Pose Estimation using Depth Merging Intersection-Based Method and Visible Point Cloud Template Alignment

Zero-shot 6D object pose estimation plays a crucial role in robotic manipulation, particularly for grasping unseen objects in complex environments. One of the leading frameworks in this domain is SAM-6D, an end-to-end instance segmentation and pose estimation model for predicting 6D object poses. In this work, we propose two key contributions to enhance the performance of SAM-6D. First, we introduce the Merging Intersection-Based Method, which incorporates depth information to refine instance masks by processing the depth image, evaluating mask intersections, and merging over-segmented regions. This method reduces over-segmentation by leveraging geometric cues from depth, resulting in a mean IoU improvement of 0.75%. Second, we propose the Visible Template Point Cloud Alignment (VPTA) method, which improves pose refinement by aligning only the visible surfaces of the rendered object model with the segmented point cloud from the input image. Together, these methods significantly enhance both segmentation quality and pose estimation accuracy. When combined, they yield an ADD/ADD-S reduction of 20.8% and an AUC gain of 2.14%. These enhancements contribute to more reliable zero-shot 6D object pose estimation, enabling more robust robotic interaction with novel objects in complex, real-world environments.