LFCNet: Deep Learning-Based LiDAR-Fisheye Camera Online Automatic Targetless Extrinsic Calibration

This paper presents LFCNet, an end-to-end deep-learning framework for targetless extrinsic calibration between LiDAR and fisheye cameras. The method is specifically designed for autonomous driving systems, where accurate alignment of sensor data is critical. LFCNet performs calibration without physical targets and operates online, which makes it practical for real-time applications involving fisheye cameras and LiDAR. Sensor fusion has become a key strategy to improve the perception capabilities of autonomous vehicles. Among the various sensors used, LiDAR and cameras are commonly employed due to their complementary strengths. Fisheye cameras, in particular, provide a wide field of view and capture more comprehensive visual information. As a result, precise calibration between these sensors is essential to align their outputs within a unified coordinate system and maintain consistent environmental understanding. Conventional calibration methods often rely on statistical techniques, which struggle with high-dimensional multivariate data. Deep learning-based approaches have recently shown promise, particularly for LiDAR–camera systems. However, calibration involving fisheye cameras remains underexplored. LFCNet addresses this gap through a lightweight model with 12.1 million parameters, and a novel two-stage feature-extraction architecture that captures both local and global features while compensating for fisheye distortion. Experimental results demonstrate that LFCNet achieves high accuracy, with a mean calibration error of 0.509 cm in translation and 0.063° in rotation—setting a new benchmark for real-time, learning-based LiDAR–fisheye calibration. The code for LFCNet is available at https://github.com/rangganast/LFCNet