End-to-end Automatic Targetless Calibration between Low-resolution LiDARs and Cameras based on Cross-modal Attention

LiDARs and cameras, commonly used sensors on autonomous vehicles and robots, require precise calibration for accurate joint environmental perception. Current calibration methods, capable of adjusting both intrinsic and extrinsic parameters, often depend on physical targets or human intervention, making them unsuitable for automatic online calibration. Moreover, these methods are primarily optimized for dense point clouds, as they are often tuned on open datasets featuring high-quality point cloud data. However, low-resolution LiDARs, preferred for their cost-effectiveness, are prevalent in practical applications. To automate the calibration process while maintaining high precision on sparse point clouds, multi-modal image registration techniques are leveraged to develop an end-to-end, cross-attention-based neural network for automatic calibration of low-resolution LiDARs and Cameras. Specifically, given an initial perturbation on point clouds, our network is capable of recovering the true extrinsic parameters without manual intervention. Additionally, specialized input preprocessing and training strategies are incorporated to enhance performance with sparse point clouds. Promising results are achieved on both open datasets and custom sensor acquisitions, highlighting the effectiveness and adaptability of our network to diverse scenarios.