Line-Based Targetless Camera–LiDAR Calibration via Extrinsic Initialization and Multiconstraint Driven Mixture Model

Targetless calibration is a prominent research area in multisensor fusion systems. Existing methods typically rely on a known initial guess, and the discrepancies among sensor modalities lead to matched features being corrupted by outliers, significantly impacting calibration accuracy. To overcome these limitations, we propose a novel method for camera–LiDAR calibration. Our approach first leverages the orthogonal dominant directions (DDs) and lines from a single scene to obtain the initial pose. Furthermore, to handle outliers, we introduce an effective line matching approach called the multiconstraint driven mixture model (MCMM). This model leverages collinear constraints to improve outlier removal capability. To enhance flow space smoothness, we enforce smooth flow constraints on the assumed correspondences in an arc-based reproducing kernel Hilbert space (RKHS). Local constraints are also introduced to preserve the triangular topological structure among neighboring vector arcs generated by inliers. The unknown variables of the MCMM are solved by the expectation–maximization (EM) algorithm. Subsequently, given line correspondences from MCMM, the calibration parameters are estimated through an iterative refinement. Simulation and real experiments demonstrate that the proposed method provides superior performance. Comparative experiments utilizing nonrepetitive scanning LiDAR qualitatively and quantitatively indicate that our method outperforms other targetless approaches, achieving an average error of 0.8 and a variance of less than 0.7.