Instance-Aware Spatial-Frequency Feature Fusion Detector for Oriented Object Detection in Remote-Sensing Images

In recent years, fusing multitype features poses great potential for oriented object detection (OOD) in remote-sensing images (RSIs). Due to the inexplicit operation of modeling orientation variations, convolutional neural networks (CNNs) are difficult to perceive objects under different transformations (angles and scales). In this article, we propose a novel instance-aware spatial-frequency feature fusion detector (SFFD) for OOD in RSIs. First, a layerwise frequency-domain analysis (L-FDA) module is built along with CNN layers to extract frequency features. Getting rid of the constraints such as horizontal rectangular kernels in CNNs, our L-FDA possesses an outstanding ability to locate mutational signals from frequency space. These mutational signals record the scale and angle information of the oriented instances in images. Subsequently, CNN and frequency features are sent into the region-of-interest (RoI) Pooling layer to obtain multitype instance-level RoI features. Moreover, the proposed instance-aware cross-feature fusion (CFF) module explores the interaction between these diverse features which provides an explicit indicator to compensate for the orientation information ignored by instance-level CNN features. Finally, our SFFD unifies the proposed L-FDA module and CFF module into the detection network to localize oriented instances in RSIs. We compare our method with many state-of-the-art methods on DOTA, HRSC2016, and NWPU VHR-10 datasets. Experimental results verify the validity of modeling instance-level object relations from the frequency domain and CNNs for OOD.