Low-Light and Infrared Multimodal Remote Sensing in Nighttime Rescue Mission: A Review of Anomaly Detection Methods

When facing natural disasters such as sudden floods at night, due to sudden natural disasters, high timeliness of rescue information, and complexity and diversity of rescue environment, it is difficult for ground personnel to perform rescue operations in disaster area in time. Remote sensing uncrewed aerial vehicle (UAV) technology plays an escalating role in disaster relief due to fast response and high flexibility advantages. However, with low nighttime visibility level, complex post-disaster environment, and numerous obstructions, traditional UAV’s visible light remote sensing struggles to achieve accurate rescue detection at night. Therefore, the multimodal detection methods are investigated using low-light and infrared modalities, exploring the integration of data fusion and detection in night rescue applications, and examining the advantages and disadvantages of different anomaly detection methods here. This article provides the following contributions: 1) a fully-annotated low-light infrared co-observation multimodal remote sensing image dataset for nighttime emergency rescue detection, termed MRSI-NERD; 2) a benchmark test for most state-of-the-art unsupervised anomaly detection methods to thoroughly explore their capability in extracting useful information from normal samples; and 3) a low-light infrared bimodal fusion method based on frequency-domain feature decomposition, which enhances the performance of detectors. The performance of eight types of traditional or deep learning-based detection methods on the MRSI-NERD dataset is reported, including metrics such as receiver operating characteristic (ROC)-area under the curve (AUC), false positive rate (FPR), and true positive rate (TPR). Additionally, a comprehensive analysis of the principles and performance of each category of detection methods is provided.