Pixel-Level Domain Adaptation for Real-to-Sim Object Pose Estimation

Transferring robotic grasping skills learned from a simulator to the real world is beneficial in reducing the cost of labeling. However, the models trained on synthetic data are brittle when being applied to real-world data due to the domain gap. In this article, we propose cycle-consistency, content-consistency, and mapping-consistency (CCM) pixel-level domain adaptation (Pixel-DA), a novel real-to-sim approach to unsupervised domain adaptation for object pose estimations, which outperforms conventional domain adaptation methods in preserving structural information, semantic information, and object pose during the transfer. The pipeline decouples domain adaptation and pose estimation, which allows the CCM Pixel-DA method to be integrated into state-of-the-art object pose estimation networks. The proposed method is further integrated into a pipeline for robot grasping. Experimental results on a real-world robot grasping system validate that the system is capable of grasping real-world objects without object pose annotations in the real-world domain.