Off-Road Freespace Detection with LiDAR-Camera Fusion and Self-Distillation

LiDAR-camera fusion has gradually become the mainstream for the freespace detection in unstructured off-road environments. However, existing methods mainly use the traditional method to densify the sparse LiDAR data in the perspective view, which introduces noise and limits the representation ability. In this paper, we propose a lightweight end-to-end freespace detection network with cascaded LiDAR-camera fusion and multi-scale self-distillation. It first performs sparse freespace detection in the range view, and then projects the range-view features onto the perspective view and densifies them. The dense features obtained are fused with camera images to get the final freespace detection results. In our method, the cascaded fusion strategy reduces the impact of resolution differences between LiDAR point clouds and camera images, and the introduction of noise during the data densification process. The multi-scale self-distillation strategy distills knowledge from the LiDAR-camera fusion module to the perspective-view module to further improve the freespace detection performance using LiDAR data only. Experiments on the off-road ORFD datasets demonstrate the effectiveness of the proposed cascaded fusion and multi-scale self-distillation strategies, our method obtains 93.4% IoU at speeds of more than 50 Hz. It also achieves state-of-the-art performance among all LiDAR-based freespace detection methods.