Visible-Infrared Features Fusion Based Object Detection

Fusion techniques are frequently utilized in the realm of multimodal object detection tasks. While many current studies showcase their proficiency in generating visually pleasing fused images, only a limited number of them focused on the object detection performance. This study addresses the issue by presenting an end-to-end framework for object detection through the fusion of visible and infrared features (VIFF). Specifically, our approach involves the use of two distinct processing units that independently extract features from visible and infrared images, followed by the fusion of these features using a novel fusion strategy. While the visible feature processing unit preserves the direction of the gradient of visible images, the infrared feature processing unit focuses on extracting the contrast and semantic features of infrared images. Both features are aggregated by attention mechanisms and then fed into the backbone of the object detection networks. Our fusion network achieved superior object detection accuracy compared to existing state-of-the-art approaches on various datasets. We have also demonstrated that the proposed visible feature and infrared feature processing units are capable of enhancing the performance of various object detection models.