FDENet: Fusion Depth Semantics and Edge-Attention Information for Multispectral Pedestrian Detection

In recent years, multispectral pedestrian detection as a multimodal solution, using visible light and thermal image multimodal information to solve pedestrian detection challenges in low light environments, plays a crucial role. However, most existing methods directly mix the results of the two modalities or by simply integrating the two modality features. This fusion strategy ignores some features that have specific meaning and are important to the detection problem, and lacks reliable comprehensibility. Drawing inspiration from this insight, in this letter, we propose a pedestrian detection method based on the fusion of multimodal edge information extraction and global semantic consistency, aiming to achieve more full use of edge information-assisted guidance multimodal for accurate fusion detection. We first build an edge attention operation module (EAO) to extract pedestrian edge features, and then introduce a global semantic consistency module (GSC) to make full use of edge features to stimulate guidance modality fusion, and improve the modality consistency between visible light and thermal information. By using challenging KAIST and CVC-14 benchmarks, we observed significant improvements compared to previous state-of-the-art methods. Our proposed method not only outperformed existing techniques but also achieved remarkable accuracy and robustness in the challenging domain of pedestrian detection.