PSNet: LiDAR and Camera Registration Using Parallel Subnetworks

The working environment of autonomous driving and robot navigation is so complex and dynamic that a single type of sensor is insufficient for performing object detection. Thus, in many perception schemes, the LiDAR-camera fusion strategy is preferred. However, the performance of a LiDAR-camera fusion heavily relies on a set of accurately calibrated extrinsic parameters. We propose PSNet, an end-to-end convolutional neural network (CNN) for calibration; this is the first calibration network to use parallel subnetworks to obtain multiresolution features and fuse them adaptively to encourage robustness against different initial error ranges. The method has three key characteristics: (i) Addition of a downsampling block to improve suitability for sparse projected depth maps; (ii) Connection of the high-to-low resolution convolution streams in parallel to obtain multiresolution features that are spatially more precise and contain richer semantic information; (iii) Fusion of multiresolution streams by the multiscale feature aggregation module. The network corrects errors from initial calibration to the ground truth online, rather than directly obtaining the accurate parameters. We evaluated our model on the KITTI datasets and it outperformed other CNN-based methods. In addition, extensive experiments evaluating the model with untrained and unfamiliar datasets demonstrated that our method exhibited good generalization ability.