Fast Partial-Modal Online Cross-Modal Hashing

Cross-Modal Hashing (CMH) has become a powerful technique for large-scale cross-modal retrieval, offering benefits like fast computation and efficient storage. However, most CMH models struggle to adapt to streaming multimodal data in real-time once deployed. Although recent online CMH studies have made progress in this area, they often overlook two key challenges: 1) learning effectively from streaming partial-modal multimodal data, and 2) avoiding the high costs associated with frequent hash function re-training and large-scale updates to database hash codes. To address these issues, we propose Fast Partial-modal Online Cross-Modal Hashing (FPO-CMH), the first approach to tackle online cross-modal hash learning with partial-modal data. This marks a significant shift from previous methods that rely on fully-available multimodal data. Specifically, our approach introduces a multimodal dual-tier anchor bank, initialized using offline training data, which allows offline-trained CMH models to adapt seamlessly to partial-modal data while progressively updating the anchor bank. By leveraging gradient accumulation and asynchronous optimization, FPO-CMH facilitates efficient online cross-modal hash learning. Additionally, an initial-anchor rehearsal strategy is employed to prevent model catastrophic forgetting during online optimization, ensuring the code invariance of database hash codes and eliminating the need for frequent hash function re-training. Extensive experiments validate the superiority of FPO-CMH, especially in handling streaming partial-modal multimodal data, a more realistic scenario. The source codes and datasets are available at https://github.com/DandelionWow/FPO-CMH