SED-DETR: A Scale-Enhanced Deformable Detection Transformer for Remote Sensing Images

Detection transformer (DETR) has emerged as a highly promising approach in object detection (OD) and has attracted significant interest. However, most DETR-like methods cannot simultaneously leverage the shape and scale priors for attention calculation, resulting in limited performance in detecting remote sensing objects with diverse shapes and scales. To address this issue, this article proposes a scale-enhanced deformable DETR (SED-DETR) for remote sensing OD (RSOD). The core component of SED-DETR is SED attention (SEDA), which is designed based on the principles of deformable shape and dynamic scale. Specifically, the SEDA module utilizes multiscale attention heads. First, conventional multiple attention heads are consolidated into several scale heads through an adaptive scale aggregation (Scale-Agg) approach, which dynamically adjusts the distributions of different scales to enhance the scale-aware modeling ability. For each scale head, dilated sampling is applied at a specific dilation rate to capture multiscale receptive fields. The sampled positions are further refined by learnable offsets predicted from query features, enabling a deformable dilated mechanism for fine-grained feature extraction of multiscale instances. Finally, we adopt the mixed query selection and the denoising training defined in DINO to implement SED-DETR. Experimental results on the xView, DIOR, NWPU VHR-10, and COCO datasets demonstrate that SED-DETR outperforms state-of-the-art (SOTA) DETR-like methods. Specifically, SED-DETR achieves 5.6%, 10.9%, and 8.6% mAP gains over the baseline deformable DETR on the xView, DIOR, and NWPU VHR-10 datasets, respectively. The source code is available at https://github.com/zzy599/SEDDETR