A Novel Nonlocal-Aware Pyramid and Multiscale Multitask Refinement Detector for Object Detection in Remote Sensing Images

Object detection (OD) is an important task of computer vision and has been widely used in many fields, including remote sensing (RS). However, the complex scenes, large-scale variation, and dense instances of RS bring huge challenges to OD. To meet these challenges, a novel Nonlocal-aware Pyramid and Multiscale Multitask Refinement Detector (NPMMR-Det) is proposed. Specifically, nonlocal-aware pyramid attention (NP-Attention) is designed for guiding a neural network model to focus more on efficient features and suppress background noise. Then a multiscale refinement feature pyramid network (MSR-FPN) is proposed to fuse the multiscale context features extracted by the NP-Attention guided neural network and adjust the optimal receptive field. In order to use these features more effectively, a multitask refinement head called MTR-Head, with offset sharing and a modulation mechanism, is developed to refine the feature misalignment between the localization task and the classification task. Extensive experiments performed on two public RS data sets demonstrate that the proposed NPMMR-Det achieves competitive performance compared with state-of-the-art methods.

I. Introduction

As a fundamental task of computer vision image analysis, object detection (OD) is widely used in many fields. The pipeline of the classic OD method is mainly based on the Deformable Part Models (DPM) [1], which has also been applied to the remote sensing (RS) field by Cheng et al. [2] and Chen et al. [3] to detect vehicles in aerial images. The DPM is a detection method using handcrafted features and sliding windows but is limited by its complexity of feature design and the inefficiency of object search.