SCREAM: SCene REndering Adversarial Model for Low-and-Non-Overlap Point Cloud Registration

Recent learning-based models excel in point cloud registration for low-overlap scenes but falter in scenarios with minimal overlap. In this article, we propose a novel method to address the extreme case of low-overlap registration: non-overlapping point cloud registration. This scenario involves input point clouds that do not have overlapping regions but are adjacent to each other after registration. While the practical application value of non-overlapping point cloud registration remains to be explored, we believe that researching this issue contributes to enhancing the performance of registration in scenarios with extremely low overlap. Abandoning conventional overlapping region detection, we directly generate the registered source point cloud with SCREAM, a generative adversarial network (GAN). The generator incorporates information from the target point cloud into the source point cloud’s features and generates the registered source point cloud. To further align the generated results with the target point cloud, we propose a differentiable renderer that renders both the target and predicted point clouds into depth maps. These depth maps are then used as inputs to a discriminator to determine whether the generated results align with the target point cloud. Rigid transformation can be directly estimated from the correspondences between the source and the generated point clouds, bypassing the need for detecting overlapping regions, feature matching, and RANSAC steps found in previous methods. Extensive experiments demonstrate that SCREAM not only outperforms common overlapping point cloud registration scenarios but also achieves a registration success rate of 52.6% for the first time in non-overlapping scenes. We also constructed a new indoor scene registration dataset, 3DZeroMatch, specifically designed to explore non-overlapping registration problems. Our code and the dataset 3DZeroMatch are accessible at https://github.com/xujiabo/SCREAM/.