Cross-Modality Channel Mixup and Modality Decorrelation for RGB-Infrared Person Re-Identification

This paper focuses on RGB-infrared person re-identification, which is challenged by a large modality gap between RGB and infrared images. Most existing methods attempt to learn discriminative modality-invariant features. These methods make use of identity annotations while they do not sufficiently exploit intra-modality and cross-modality sample relations using modality annotations. In this paper, we propose a Cross-modality channel Mixup and Modality Decorrelation method (CMMD) that explores sample relations at both image and feature levels. This method is designed to reduce redundant modality-specific information of the representations and highlight modality-shared information. Specifically, we first design a cross-modality channel mixup (CCM) augmentation at the image level, which combines a random RGB channel and an infrared image to generate a new one by mixup, while keeping identity information unchanged. This augmentation can be integrated into other methods easily without introducing extra parameters or models. In addition, modality decorrelation quintuplet loss (MDQL) is further presented to mine hard samples in a batch, that is, positive/negative intra/cross-modality samples, to learn modality-invariant representations in the shared latent space at the feature level. This loss suggests that the closest negative sample and the farthest positive sample should have an equal probability of appearing in both modalities. Comprehensive experimental results on two challenging datasets, i.e., SYSY-MM01 and RegDB, demonstrate competitive performance of our method with state-of-the-art ones.