Feature Fusion with Slim Non-Local Module for Multispectral Pedestrian Detection

Feature fusion plays a vital role in multispectral pedestrian detection system. Traditional fusion in different layers rely on simplely concat method for efficiency, but the pixel-wise spatial relationship in both short and long range are neglected. Towards this problem, non-local network provides a possible solution to cross-modality feature fusion, which is able to capture the long range relationship between pixels in the feature map from both RGB and thermal modalities. However, the naive utilization of non-local network brings significant memory footprint cost especially for the high resolution feature maps. To this end, we propose a slim non-local network, which compress the size of the attention map, leading to significant reduction of memory load. Besides, ablation studies with our slim non-local module for different stage are also investigated, which indicates that the optimal fusion schema comes from all the feature layers. Experimental results on KAIST dataset report that our method can outperform the baseline method with reasonable cost.