Enhanced Invariant Feature Joint Learning via Modality-Invariant Neighbor Relations for Cross-Modality Person Re-Identification

Cross-modality visible-Infrared person re-identification (cm-ReID) is extremely challenging due to the huge modality discrepancy between RGB and IR modalities. Existing methods focus on the sample features themselves, trying to learn modality-invariant features and perform alignment to reduce the modality discrepancy in dataset-level, while the negative impact of specific features and the identity optimization are not specifically addressed. Moreover, most methods that only extracts modality-invariant appearance features cannot acquire enough discriminative matching information for identifying different persons since the information of invariant features is limited compared with original features. Accordingly, in this paper, we propose a Enhanced Invariant Feature Joint Learning Framework (EIFJLF) for cm-ReID to handle the above problems. First, we propose a specific feature confusion baseline with a novel channel-blended transformation, which confuses the visible color and infrared spectrum to alleviate the influence of specific features, so that model pays more attention to other discriminative invariant features. Second, we present an adaptive heterogeneous center loss for better identity optimization. The adaptive margin of the loss makes samples not too close to the center, avoiding losing effectiveness too early and overfitting meantime further boosting performance. Finally, we design a novel similarity feature refinement module to utilize intra-modality relations and achieve invariant information compensation. Intra-modality relations are valuable built-in invariant features and we model these relations with similarity between samples into affinities and then update the original features to achieve information compensation. EIFJLF works for more informative invariant feature learning and more stable alignment. For cm-ReID, our work is a brand new attempt. Extensive experimental results on two standard benchmarks have demonstrated superiority of the proposed method compared with state-of-the-art methods.