Multi-modality Cascaded Fusion Technology for Autonomous Driving

A highly reliable sensor is crucial for autonomous driving, which draws more attention on multi-modality fusion. This paper proposes a general multi-modality cascaded fusion framework, exploiting the advantages of decision-level and feature-level fusion, utilizing target position, size, velocity, appearance and confidence to achieve accurate fusion results. In the fusion process, dynamic coordinate alignment(DCA) is conducted to reduce the error between sensors from different modalities. In addition, the calculation of affinity matrix is the core module of sensor fusion, we propose an affinity loss that improves the performance of deep affinity network(DAN). Last, compared to the end-to-end fusion methods, our step-by-step cascaded fusion framework is more interpretable and flexible. Extensive experiments on Nuscenes [1] dataset show that our approach achieves the state-of-the-art performance.