Multiscale Multilevel Residual Feature Fusion for Real-Time Infrared Small Target Detection

Detecting infrared dim and small targets is one crucial step for many tasks such as early warning. It remains a continuing challenge since characteristics of infrared small targets, usually represented by only a few pixels, are generally not salient. Even though many traditional methods have significantly advanced the community, their robustness or efficiency is still lacking. Most recently, convolutional neural network (CNN)-based object detection has achieved remarkable performance and some researchers focus on it. However, these methods are not computationally efficient when implemented on some CPU-only machines and a few datasets are available publicly. To promote the detection of infrared small targets in complex backgrounds, we propose a new lightweight CNN-based architecture. The network contains three modules: the feature extraction module is designed for representing multiscale and multilevel features, the grid resample operation module is proposed to fuse features from all scales, and a decoupled head to distinguish infrared small targets from backgrounds. Moreover, we collect a brand-new infrared small target detection dedicated dataset, which consists of 68311 practical captured images with complex backgrounds for alleviating the data dilemma. To validate the proposed model, 54758 images are used for training and 13553 images are used for testing. Extensive experimental results demonstrate that the proposed method outperforms all traditional methods by a large margin and runs much faster than other CNN methods with high precision. The proposed model can be implemented on the Intel i7-10850H CPU (2.3 GHz) platform and Jetson Nano for real-time infrared small target detection at 44 and 27 FPS, respectively. It can be even deployed on an Atom x5-Z8500 (1.44 GHz) machine at about 25 FPS with
$128\times128$
local images. The source codes and the dataset have been made publicly available at
https://github.com/SeaHifly/Infrared-Small-Target
.