Robust Indoor Person Re-Identification With Multimodal Training

Existing person re-identification (ReID) methods mainly rely on images and videos to match persons across cameras, yet visual data captured by cameras are vulnerable to environmental interferences (e.g., illumination and occlusion) or personal appearance changes, leading to performance degradation under such scenes. Meanwhile, the popularization of Wi-Fi networks has allowed probe requests to be captured for mobile sensing applications, such as crowd counting and trajectory estimation. However, the MAC address randomization technique adopted by modern devices breaks the association of probe requests and adversely affects the functionality of these applications. In this article, we propose MaRPA, the first multimodal training approach that incorporates both videos and Wi-Fi probe requests to simultaneously promote tasks of probe requests association and person ReID. MaRPA first distinguishes among pairwise probe request frames through a contrastive learning model. It then matches video and probe request sequences by exploring their similarities from the position and the vision aspects. Matched videos and probe requests provide complementary information and generate more robust features for both tasks. To evaluate MaRPA, we contribute a new dataset containing synchronous videos and probe requests data for probe requests association and person ReID. Experimental results demonstrate the effectiveness of our approach. For probe requests association, it achieves >85% discrimination accuracy and >0.90 V-measure score; for person ReID, it achieves 75.8% mean average precision and 90.6% Rank-1, improving state-of-the-art video-based ReID methods by over 40%.