Learn to Search a Lightweight Architecture for Target-Aware Infrared and Visible Image Fusion

Deep learning technology has recently achieved remarkable progress in infrared and visible image fusion. Nevertheless, existing methods have encountered blurred targets or unfaithful textural details on their fused results. They suffer from high computational expenses, thus unable to directly serve the subsequent high-level vision tasks. In this letter, to alleviate this issue, we proposed leveraging a lightweight architecture based on Neural Architecture Search (NAS) to realize the infrared and visible image fusion in an end-to-end manner, significantly reducing the computational expenses and runtime. Concretely, we construct a search-based architecture to explore the feature representation across different modalities automatically. Then a saliency-based loss function is designed to retain both the distinct target and texture details. Motivated by the cooperative principle, we also formulate a flexible hardware-sensitive regularization constraint in our loss function for discovering efficient operations. As a result, we can generate a target-distinct fused result with high efficiency. Extensive qualitative and quantitative experiments reveal that our method has superior performance against the state-of-the-art methods, especially highlighting the target, retaining realistic details, and achieving fast running speed. Specifically, our method increases by 150% in time, reduces the FLOPS by 21.3% and reduces the model parameters by 25%.