RecNet: Reinforcement Common Feature Mapping Network for Fast Template Matching in Visible-LWIR Images

We present RecNet, a novel end-to-end CNN approach for fast template matching in cross-spectral images, addressing nonlinear intensity disparities and appearance differences in ground-level imagery through a simple yet effective design. Our key innovation lies in the seamless integration of Zero-mean normalized cross correlation (ZNCC), which is well-validated for handling nonlinear intensity variations, with CNN-based shape difference learning. Unlike aerial imagery, ground-level cross-spectral matching presents unique challenges due to limited common features, which RecNet effectively addresses. Through comprehensive experiments, including layer-wise visualization analysis and comparative studies with pooling layer combinations, we validate our architecture’s effectiveness. Experiments on KAIST Pedestrian, Log-Gabor Histogram Descriptor(LGHD), and Road Scene datasets demonstrate RecNet’s superior performance and real-time capabilities compared to state-of-the-art methods. Additional evaluations using KAIST nighttime imagery and the M3FD dataset verify RecNet’s generalization capabilities and stability across diverse scenes and conditions, while also identifying limitations and future research directions.