Modality-Independent Regression and Training for Improving Multispectral Pedestrian Detection

Multispectral pedestrian detection enables an all-weather detector by fusing the complementary information of RGB and thermal images. However, due to the modality difference and the spatial misalignment between RGB and thermal sensors, the positions of the same pedestrian in two different sensors may be different and some pedestrians may only be visible in one sensor. The existing methods mostly adopt unified annotations to supervise different modalities, this is not conducive to the learning of different modal features. Therefore, we propose a multispectral pedestrian detector based on MBNet with modality-independent regression modules, named MBPNet. It adopts the corresponding modality-independent annotation to train the regression branch for different modalities, separately. Then, a decision-level fusion strategy is introduced to fuse the regression results from different branches. Experiments show that the proposed method outperforms the state-of-the-art methods on the KAIST pedestrian detection dataset.