Cognition-Inspired Dynamic Feature Integration Network for RGB-D and RGB-T Salient Object Detection

Salient object detection (SOD) is widely used in quality inspection scenarios, such as rail surface detection. Recent studies have proven that incorporating complementary information like depth and thermal images is conducive to SOD. Effectively leveraging the advantages of each modality while eliminating intermodality noise in multilevel fusion has been a research hotspot. Most existing works use convolution and attention mechanisms for modality interaction but overlook semantic similarity during fusion, leading to poor performance in some challenging scenarios. In this article, inspired by psychology studies on the human visual system (HVS), we propose a dynamic feature integration network (DFINet) that simulates the correlation mechanism between human attention and semantics for RGB-D and RGB-T SOD. Specifically, to better capture the modal-specific features in semantics, we first employ a multigranularity-based presegmentation method, namely, the presegmentation injection module (PIM), to enhance and preserve the modal-specific features at different layers of the backbone network. Then, a dynamic feature fusion module (DFFM) is devised to simulate the mechanism of HVS where specific semantic regions gain more attention. This module evaluates the semantic similarity between different modal features and determines the weights for each modal feature in the fusion. The encoded multimodal features are fed into a staircase decoder which can retain deep semantic information to boost accuracy. Extensive experiments on RGB-D and RGB-T SOD datasets validate that our proposed cognition-inspired framework has excellent and competitive performance with good generalization and robustness.