LiDAR-Camera Cross Fusion Network Toward 3-D Object Detection in Self-Driving

In recent years, the fusion of LiDAR and camera by means of deep learning has attracted much attention and made great progress in 3-D object detection. But feature receptive field of LiDAR points in 3-D space and that of pixels in the image plane are not consistent in size in deep learning process; this affects the LiDAR-Camera fusion quality. To alleviate this problem, we propose a cross fusion network (CF-Net) in this article to automatically choose an image feature of the appropriate receptive field to get fused with point cloud feature. Specifically, we input image features of all scales to the fusion module, and the weight of each image feature is calculated by our designed attentive point-image fusion (APIF) module to control their contribution proportion. In addition, different from most point-based 3-D object detection works, we adopt a semantic sampling strategy to replace commonly used farthest point sampling with the assistance of image features, which retains more foreground points in the stage of point cloud downsampling. We conduct experiments on KITTI and nuScenes datasets and achieve advanced performance in 3-D object detection benchmark. Comparative experiments with advanced fusion-based and point-based works demonstrate the excellent performance of our work.