Line-Based 2-D–3-D Registration and Camera Localization in Structured Environments

Accurate registration of 2-D imagery with point clouds is a key technology for image-Light Detection and Ranging (LiDAR) point cloud fusion, camera to laser scanner calibration, and camera localization. Despite continuous improvements, automatic registration of 2-D and 3-D data without using additional textured information still faces great challenges. In this article, we propose a new 2-D-3-D registration method to estimate 2-D-3-D line feature correspondences and the camera pose in untextured point clouds of structured environments. Specifically, we first use geometric constraints between vanishing points and 3-D parallel lines to compute all feasible camera rotations. Then, we utilize a hypothesis testing strategy to estimate the 2-D-3-D line correspondences and the translation vector. By checking the consistency with computed correspondences, the best rotation matrix can be found. Finally, the camera pose is further refined using nonlinear optimization with all the 2-D-3-D line correspondences. The experimental results demonstrate the effectiveness of the proposed method on the synthetic and real data set (outdoors and indoors) with repeated structures and rapid depth changes.