Targetless Extrinsic Calibration of Camera and LiDAR via Identifying True 2-D–3-D Line Matching Among Unknown Line Correspondences of Structured Environments

For those systems consisting of cameras and LiDAR, extrinsic calibration is unavoidable. Traditional methods usually start from a known initial pose and alternately optimize the extrinsic parameters of these sensors. Their main limitations are that a good initial guess is required and matching features are inevitably corrupted by outliers, which affect the accuracy of calibration significantly. In this article, we present a novel approach to estimating the extrinsic parameters between the LiDAR and the camera. We first obtain the initial transformations by leveraging the structural regularity such as the 2-D–3-D dominant directions and 2-D–3-D parallel line pairs of the structured scenes, then achieve 2-D–3-D line registration using the nearest neighbor approach. Furthermore, to handle outliers, we propose an effective line feature matching approach, termed as local triangular preserving (LTP) matching, the major objective of which is to maintain the triangular topological consistency among neighboring vector arcs generated by inliers. We formulate it into a mathematical model and derive a simple closed-form solution. Then, the calibration parameters are estimated by using correspondences from LTP and further refined by the Levenberg-Marquardt (LM) algorithm with an outlier removal strategy. Experimental results show that our method can achieve better performance than state-of-the-art methods.