Metal Parts’ Zero-Shot 6D Pose Estimation via Foundation Model and Template Update for Industrial Scenario

The 6D pose estimation for metal parts is essential in industrial robotic applications. Although the 6D pose estimation methods that rely on object-specific training have gained extensive concern, these methods cannot generalize to novel objects. Recent novel object pose estimation methods are solving this issue using task-specific fine-tuned CNNs for deep template matching. However, these methods require expensive training procedures and do not consider pose distribution of rendered templates. Recently, foundation models show strong representation learning ability and can encode both the high spatial information and the semantic information. In this study, we present a metal parts’ zero-shot 6D pose estimation method by foundation model without retraining and incorporate prior information of metal parts’ poses to generate rendered templates that align with the pose distribution in real world. DINOv2, a recent vision foundation model with impressive generalization capabilities, is used for matching rendered templates against query images of metal parts. In addition, we introduce the optimal transport (OT) as a similarity metric. Then, the GlueStick is used to establish local keypoint correspondences, which enable deriving geometric correspondences and are used for estimating the metal part’s 6D pose with PnP/RANSAC. Given an industrial scenario, we first estimate metal parts’ pose, select, and save high-confidence pose results to pose buffer, which is used to update the templates. The experiments on the MP6D and ROBI datasets showcase that the proposed method has a better performance than MegaPose. We also conduct real-world experiments, which demonstrate the robustness of the proposed method. Code is available at https://github.com/sunhan1997/IndusPose