Iterative Saliency Aggregation and Assignment Network for Efficient Salient Object Detection in Optical Remote Sensing Images

Motivated by the pursuit of efficient salient object detection in remote sensing, researchers have devoted considerable efforts to devising lightweight models due to low running efficiency of the cumbersome models. Although the existing lightweight models have made impressive progress in improving efficiency, there is often a notable sacrifice in inference accuracy, making it challenging to attain both high-quality output and high efficiency. In this article, we propose an iterative saliency aggregation and assignment network (ISAANet) to reconcile the dilemma of balancing accuracy and efficiency. ISAANet adopts a recurrent architecture with bidirectional communication between the encoder and the decoder to boost detection performance. Specifically, a saliency aggregation mechanism is used to generate complementary information by integrating multilevel saliency cues, corresponding to the feedforward information flow from the encoder to the decoder. As for the feedback flows from the decoder to the encoder, a saliency assignment approach is proposed to inject the region and edge prompts into the encoder for hierarchical feature updates, emphasizing the representation of salient information. The encoder is progressively reinforced by alternating iterations of saliency aggregation and assignment, which further helps the decoder to produce more reliable detection results. Moreover, the effective fusion of cross-level features is key to realizing information complementarity and enhancing aggregation quality. However, conventional fusion schemes often neglect feature importance disparities, which can lead to noise accumulation. To address this issue, we introduce a hierarchical interaction and dynamic integration (HIDI) module for feature fusion. The HIDI module groups fusion features and dynamically adjusts their weights based on their information quality, which suppresses noise interference and strengthens the model’s learning of meaningful features. The experimental results demonstrate that the proposed ISAANet not only effectively compensates for the accuracy deficiency of the existing lightweight models but also achieves a faster running speed of 108 FPS, which shows a favorable tradeoff between efficiency and accuracy. The code will be available at https://github.com/YiuCK/ISAANet and https://openi.pcl.ac.cn/OpenVision/ISAANet.