Thermal-Infrared Remote-Target Detection System for Maritime Rescue Using 3-D Game-Based Data Augmentation With GAN

This article proposes a deep learning-based thermal-infrared (TIR) remote target detection system for maritime rescue with a self-collected real TIR dataset and corresponding data augmentation method based on generative adversarial network (GAN). We have collected and established a real field TIR dataset consisting of multiple scenes imitating actual human rescue scenarios using a TIR camera (FLIR M364C). In addition, synthetic TIR data from a game (ARMA3) to augment the real TIR data are further collected to address dataset scarcity and improve the model performance. However, a significant domain gap exists between the real and synthetic TIR datasets. Hence, a proper domain adaptation (DA) algorithm is essential to overcome the gap. We suggest a target-background separation (TBS) scheme during the DA to mitigate this gap while preserving the shapes and locations of the small-size targets even after the domain transfer. Furthermore, a fixed-pattern kernel module inserted at the network front is proposed to improve the signal-to-noise ratio (SNR) as TIR remote targets inherently suffer from unclear boundaries and heavy clutters. The experimental results reveal that the segmentation network trained on both real and domain-translated synthetic TIR data shows improved performance compared to that trained on only real TIR data. Moreover, the segmentation network with the fixed-weight (FW) kernel module shows better performance than state-of-the-art methods in terms of every evaluation metric.