PFEMEF-DET: All-Weather Surveillance Detector Based on Progressive Feature Enhancement and Mutual Enhancement Fusion With Visible and Infrared Sensors

In complex environments such as night, foggy weather, and camouflage, the use of a single sensor for monitoring often faces challenges in comprehensively capturing scene information. The application of multiple sensors significantly enhances environmental perception. For instance, the combination of visible light cameras and infrared sensors is regarded as an effective fusion for this purpose. However, due to disparities among inputs from different sensors, integrating the information from these two sensors and applying it for all-weather monitoring poses certain challenges. To address this issue, this study proposes a multi-source input detection algorithm that combines infrared sensors and visible light cameras to tackle the issue of low detection accuracy of single sensors in complex and dynamic environments. Firstly, a Progressive Feature Enhancement (PFE) module is designed to enhance features that gradually decay during the network transmission process. Secondly, an Mutual Enhancement Fusion (MEF) module is constructed to achieve effective fusion of multi-source features. Finally, these two modules are embedded into a dual-stream network. Experimental results conducted on publicly available FLIR and LLVIP datasets demonstrate that compared to using a single sensor and simple concatenation fusion of dual sensors, this study's algorithm shows an improvement of 13.9/7.8 percentage points in mAP@50:95. Comprehensive experimental results indicate that the proposed modules in this study exhibit characteristics such as lightweight efficiency, ease of integration, and ease of use.