Closing the Domain Gap for Cross-modal Visible-Infrared Vehicle Re-identification

Traditional vehicle re-identification (ReID) approaches, based on visible spectrum data achieve high performance, but have limited capability of real-life applications, as they perform poorly under occluded visibility conditions, such as night-time and bad weather. In such cases, the use of infrared spectrum thermal imagery offers complementary and persistent information when the visual data contribution is inadequate. It is therefore highly beneficial to create a vehicle ReID framework that can exploit both modalities, if available, and is able to apply cross-modality matching, when ReID is required across single modal sensors. This is an extremely challenging task because the nature of the two data modalities induces high discrepancy between the two domains. In this paper we propose a robust end-to-end 2-stream vehicle ReID system that aims to solve the multi-modal and cross-modal ReID problem together by minimising the domain shift between infrared and visible distributions. Our framework consists of a shared network part, following the 2 independent streams, to extract shareable features, along with a domain alignment technique to narrow the gap between the two domains and inter-modality learning to address the cross-domain matching problem. The proposed system achieves state-of-the-art results on RGBN300 dataset, when both modalities are available at inference time. Moreover, our work is the first to explore the cross-modal settings for vehicle ReID and attempts to reduce the performance drop of the cross-modal scenario, when the query and the gallery images come from different modalities. We first measure the baseline cross-modal performance, and then prove that the proposed method improves up to 11% in mAP and 16% in Rank-1 score against the baseline.