Precise Camera–LiDAR Extrinsic Calibration Based on a Weighting Strategy Using Analytic Plane Covariances

Planarity is a practical feature for estimating the camera–LiDAR extrinsic parameters. LiDAR directly measures the 3-D plane in the form of a point cloud, but the camera captures the 2-D plane image which requires perspective-n-point (PnP) techniques for 3-D plane estimation. Likewise, these 3-D plane acquisition processes by LiDAR and camera are quite different; hence, their plane covariances from each sensor also differ. The plane measurement covariance originates from the LiDAR measurement noise; it can be directly calculated from a 3-D point cloud. The plane parameter covariances originate from the corner point errors in a 2-D image; however, their direct formulas have not been studied. In this study, we analytically derive the plane parameter covariances under the assumption of linear error propagation. To validate the proposed covariances, we implemented a simulation to observe how effectively the proposed covariances reflect on the true plane parameter covariances. Next, we validated that the proposed covariances could increase the performance of the camera–LiDAR extrinsic calibration. We compared our method with other calibration methods by varying the pixel noise levels and the number of board measurements in calibration simulations. In calibration field tests, we calibrated a scanning system equipped with a Ladybug5+ camera and a VLP-16 LiDAR and compared our method with other calibration methods.