Research of the Information Fusion Technique for Millimeter Wave Radar and Camera

Intelligent assisted driving heavily relies on accurate environment perception, a vital component for ensuring road safety. In real-world scenarios, relying solely on a single sensor or multiple sensors can be insufficient to effectively perceive the surrounding environment. Therefore, there is a growing need to harness the power of multi-source sensor modalities and integrate their data. In this context, the combination of millimeter-wave radar and camera sensors in vehicles has emerged as a prominent choice for perceiving upfront information, allowing for comprehensive analysis and comparison of different sensor options. Prior to aligning the multi-source data in space and time, it is essential to calculate the precise coordinate relationship between these sensors. Subsequently, the signals are processed using the advanced global nearest neighbor matching algorithm, synchronizing the two signals and fusing the matched targets using a weighted fusion method. To further enhance accuracy, the adaptive extended Kalman filter method efficiently tracks both matched and unmatched targets, ultimately determining the final state. Experimental outcomes strongly support the effectiveness of this fusion method in overcoming the limitations of individual sensors, thereby significantly improving target identification rates in intelligent assisted driving systems.