Online Robust Specific and Consistent Hashing

Most of the existing cross-modal hashing (CMH) methods are trained in a batch-based manner, which is time-consuming and unable to handle streaming data. Recently, online CMH methods have attracted increasing attention. However, existing online CMH methods face two limitations: 1) they usu-ally excavate the common semantic information by learning modality-specific projection matrix for each modality, while ignoring the intrinsic relationship among different modali-ties; 2) they often suffer from learning less discriminative hash codes because of insufficiently exploiting the pairwise similarity. To tackle these challenges, we propose a novel Online Robust Specific and Consistent Hashing (ORSCH) method. Specifically, ORSCH decomposes the projection ma-trices into consistent and modality-specific ones, which ef-fectively exploits intrinsic semantic information of streaming data in different modalities. Furthermore, we utilize both dis-crete and continuous labels to construct affinity matrices to improve the discrimination of hash codes. Experiments on three benchmark datasets show the superiority of ORSCH.