Feature Map Swap: Multispectral Data Fusion Method for Pedestrian Detection

This paper proposes a novel multispectral data fusion method for pedestrian detection. For all-day vision, a fusion of CCD and Infrared (IR) sensors are inevitable, and fusion of heterogeneous data based on a Convolutional neural network (CNN) is only based on the feature map concatenation of parallel CNN architecture. However, concatenation that is simply applied has a problem in that it can not fully utilize multispectral data in a deep network architecture. Therefore, this paper proposes a method called Feature Map Swap (FMS) of swapping feature maps in addition to concatenation. The proposed method can use multispectral data more efficiently by facilitating learning, by swapping the different domain weights of feature maps besides to concatenation. Also, the proposed method does not require any modifications, such as adding or removing layers to an already configured network architecture. Experimental results show that the performance of the KAIST multispectral pedestrian dataset is improved by about 7-10% based on log-average miss rate compared to simple concatenation.