External Extrinsic Calibration of Multi-Modal Imaging Sensors: A Review

With the rapid development of autonomous driving, robotics, and intelligent transportation, multi-sensor-based environment sensing technology for intelligent vehicles has become a popular research direction. In order to better fuse the data acquired by multi-sensors, accurate external parameter calibration becomes one of the critical issues. According to the method of external parameter calibration, this paper first introduces the offline calibration technology based on target and targetless methods. However, once these two methods change the relative position between the camera and the LiDAR, it can only be returned to the field to re-calibrate. The computational complexity is high, which makes it necessary to use the online calibration directly. Hence, this paper follows up with the introduction of online calibration technology based on deep learning. Unlike previous methods that need to extract features from calibration boards or environments, various types of networks can directly learn the mapping relationship between images and point clouds, From the calibration results, the average error of translation and rotation of traditional methods can reach 0.34cm and 0.45°, the average error of using deep learning networks such as LCCNet, which is the most widely used in existing networks and has good calibration effect, can reach 0.297cm and 0.017°. Compared with the traditional method, the accuracy of online calibration technology is respectively improved by 12.6% and 96.2%, which shows the results of online calibration technology are better than the traditional offline method, and there are some recently proposed methods incorporate an attention mechanism and use an optimization algorithm instead of a loss function to refine the outer parameters. From the review, learning the relative relationships between sensors through neural networks works best, and the process is relatively free of human intervention. Contrary to the existing reviews, this paper provides a general structure of calibration methods universally used in various environments and compares various methods based on this general structure.