RGB-Thermal based Pedestrian Detection with Single-Modal Augmentation and ROI Pooling Multiscale Fusion

RGB-Thermal based pedestrian detection has received more extensive attention due to the provided detailed information and thermal sensitivity of pedestrians. In this paper, a single-modal feature augmentation network (SMA-Net) is proposed. Firstly, two single-modal branches are trained separately to optimize the feature extraction of each branch in addition to the training of pedestrian detection based on fused features. Secondly, a lightweight ROI pooling multiscale fusion module (PMSF) is proposed to obtain more fine-grained and abundant features, in which pooling features of different scales are integrated by adaptively weighting. Finally, a generative constraint strategy is designed to constrain fusion by minimizing the loss function between the generated fusion image and RGB-Thermal pairs. Experimental result on the challenging dataset KAIST demonstrates that the proposed SMA-Net achieves great performance in terms of accuracy and computational efficiency.