PSD-SQ: Point Set Decoding Based on Semantic Query for Object Detection in Remote Sensing Images

Object detection in remote sensing images (RSIs) remains a challenging task due to complex variations in object scale, dense arrangements, and arbitrary orientations. Compared to the widely used multistage and one-stage approaches, query-based methods that avoid postprocessing procedures and implement end-to-end inference, have recently attracted much attention. However, existing query-based methods still face two main challenges: 1) the feature sampling regions predicted by the query vectors often fail to be aligned with the foreground features, making it difficult to accurately classify and locate potential objects; and 2) the cascade decoders are crucial for optimizing the query vectors, resulting in a slower inference process. To address the above issues, we propose a novel object detection method named point set decoding based on semantic query (PSD-SQ), which mainly consists of two components: a semantic query generator (SQG) module and an oriented point set decoder (OPSD) module. The SQG module is proposed to generate semantic query vectors with rich object information based on the semantic correlations among feature vectors. The OPSD module includes two blocks: a point sampling with angle (PSA) block and a dynamic interactor (DI) block. The PSA block is constructed to refine the sampling locations with predicted angles, aligning the sampling locations and oriented object regions, and the DI block is designed to decode the sampled features with dynamic weights, making the decoding process more efficient. The proposed method is extensively evaluated on various object detection datasets of RSIs, and the experimental results consistently demonstrate that the proposed method achieves state-of-the-art (SOTA) performance in terms of both accuracy and inference speed. In addition, our code is available at
https://github.com/I3ab/PSD_SQ
.