CSANet: Cross-Modality Self-Paced Association Network for Unsupervised Visible-Infrared Person Re-Identification

For preeminent unsupervised visible-infrared person re-identification (US-VI-ReID), existing studies typically adhere to a two-step paradigm, i.e., intra-modality clustering and inter-modality matching. Nevertheless, high intra-modality variations may result in suboptimal clusters containing intricate pedestrians, while significant inter-modality discrepancies further complicate their cross-modality associations. Most existing methods fail to adopt a differentiated approach for samples of varying difficulty, especially intricate ones. To address this, we propose enabling the model to gradually establish cross-modality associations from easy to hard, mimicking human learning patterns to avoid error accumulation caused by intricate pedestrians. To this end, we propose a Cross-modality Self-paced Association Network, termed CSANet, embracing Twain Bipartite Graph Matching (TBGM), Cross-curriculum Association Prompter (CAP) and Instance-Prototype Consistency Constraint (IPCC) modules. TBGM conceives a graph-driven metric to tailor a three-level curriculum (plain, moderate and intricate) for self-paced cross-modality learning. CAP transfers high-confidence associations deduced from the plain subsets to intricate ones, prompting exploring more complex cross-modality relationships. Alongside CAP, IPCC further enforces the intricate instances to mimic their prototype characteristics, facilitating their discriminative feature learning. Extensive experiments demonstrate CSANet’s superiority over state-of-the-art methods, highlighting the potential of self-paced learning for US-VI-ReID.