Abstract:
Current cross-modal retrieval methods heavily rely on accurate semantic labels or sample similarity measurements, and need to search for the nearest samples among all sam...Show MoreMetadata
Abstract:
Current cross-modal retrieval methods heavily rely on accurate semantic labels or sample similarity measurements, and need to search for the nearest samples among all samples in the huge search space, severely limiting the application in stratifying large-scale and high-dimensional multimodal data. To tackle with the issues, this paper proposes an unsupervised cross-modal retrieval method to bypass the semanticwise supervision and samplewise similarity from a standpoint of featurewise matching, named by unsupervised dual hashing coding (UDC). It jointly learns the dual hashing codes on semantic tagging and sample content through factorizing a feature matching potential, which is allowed to bridge the semantic and heterogeneous gaps among different modalities simultaneously through maintaining the inter-modality-consistent semantic information and cross-modality-correlated sample content. In this way, each sample is uniquely coded by a head code on semanticwise tags, and tail codes on samplewise content. The dual coding design makes it very efficient for sample retrieval, in which the query sample only need to search for the retrieved ones with the same semantic tag, greatly narrowing down the search space. The proposed model avoids the calculation of massive sample-wise similarity and works with dual hashing coding scheme, which achieves a twofold efficiency enhancement for analyzing the large-scale and high-dimensional multimodal data. Extensive experiments have been conducted to demonstrate that it achieved superiority on computational time and retrieval performance.
Published in: IEEE Transactions on Multimedia ( Volume: 26)
Funding Agency:

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Hongmin Cai (Senior Member, IEEE) received the B.S. and M.S. degrees in mathematics from the Harbin Institute of Technology, Harbin, China, in 2001 and 2003, respectively, and the Ph.D. degree in applied mathematics from The University of Hong Kong, Hong Kong, in 2007. From 2005 to 2006, he was a Research Assistant with the Center of Bioinformatics, Harvard University, Cambridge, MA, USA, and Section for Biomedical Image ...Show More
Hongmin Cai (Senior Member, IEEE) received the B.S. and M.S. degrees in mathematics from the Harbin Institute of Technology, Harbin, China, in 2001 and 2003, respectively, and the Ph.D. degree in applied mathematics from The University of Hong Kong, Hong Kong, in 2007. From 2005 to 2006, he was a Research Assistant with the Center of Bioinformatics, Harvard University, Cambridge, MA, USA, and Section for Biomedical Image ...View more

Guangdong Institute of Intelligence Science and Technology, Zhuhai, China
Bin Zhang received the B.S. and M.S. degrees from the School of Information Science and Engineering, Shandong Normal University, Jinan, China, in 2011 and 2015, respectively, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. He is currently a Postdoctoral Researcher with the Guangdong Institute of Intelligence Science and Technology, Zhuhai, C...Show More
Bin Zhang received the B.S. and M.S. degrees from the School of Information Science and Engineering, Shandong Normal University, Jinan, China, in 2011 and 2015, respectively, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. He is currently a Postdoctoral Researcher with the Guangdong Institute of Intelligence Science and Technology, Zhuhai, C...View more

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Junyu Li received the B.S. and M.S. degrees from the Guangdong University of Technology, Guangzhou, China, in 2017 and 2020, respectively. He is currently working toward the Ph.D. degree in computer science and engineering from the South China University of Technology, Guangzhou. His research interests include machine learning and image processing.
Junyu Li received the B.S. and M.S. degrees from the Guangdong University of Technology, Guangzhou, China, in 2017 and 2020, respectively. He is currently working toward the Ph.D. degree in computer science and engineering from the South China University of Technology, Guangzhou. His research interests include machine learning and image processing.View more

School of Medical Technology, Beijing Institute of Technology, Beijing, China
Bin Hu (Fellow, IEEE) was born in Beijing, China, in 1965. He received the M.S. degree in computer science from the Beijing University of Technology, Beijing, and the Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, Beijing. From 2007 to 2009, he was a Reader with the Head of Context Aware Computing Research Group, School of CTN, Birmingham City University, Birmingha...Show More
Bin Hu (Fellow, IEEE) was born in Beijing, China, in 1965. He received the M.S. degree in computer science from the Beijing University of Technology, Beijing, and the Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, Beijing. From 2007 to 2009, he was a Reader with the Head of Context Aware Computing Research Group, School of CTN, Birmingham City University, Birmingha...View more

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Jiazhou Chen (Member, IEEE) received the B.S. degree in computer science and technology from Jiaying University, Meizhou, China, the M.S. degree in software engineering from the Guangdong University of Technology, Guangzhou, China, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou. His research interests include bioinformatics, machine learning, compu...Show More
Jiazhou Chen (Member, IEEE) received the B.S. degree in computer science and technology from Jiaying University, Meizhou, China, the M.S. degree in software engineering from the Guangdong University of Technology, Guangzhou, China, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou. His research interests include bioinformatics, machine learning, compu...View more

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Hongmin Cai (Senior Member, IEEE) received the B.S. and M.S. degrees in mathematics from the Harbin Institute of Technology, Harbin, China, in 2001 and 2003, respectively, and the Ph.D. degree in applied mathematics from The University of Hong Kong, Hong Kong, in 2007. From 2005 to 2006, he was a Research Assistant with the Center of Bioinformatics, Harvard University, Cambridge, MA, USA, and Section for Biomedical Image Analysis, University of Pennsylvania, Philadelphia, PA, USA. He is currently a Professor with the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. His research interests include biomedical image processing and omics data integration.
Hongmin Cai (Senior Member, IEEE) received the B.S. and M.S. degrees in mathematics from the Harbin Institute of Technology, Harbin, China, in 2001 and 2003, respectively, and the Ph.D. degree in applied mathematics from The University of Hong Kong, Hong Kong, in 2007. From 2005 to 2006, he was a Research Assistant with the Center of Bioinformatics, Harvard University, Cambridge, MA, USA, and Section for Biomedical Image Analysis, University of Pennsylvania, Philadelphia, PA, USA. He is currently a Professor with the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. His research interests include biomedical image processing and omics data integration.View more

Guangdong Institute of Intelligence Science and Technology, Zhuhai, China
Bin Zhang received the B.S. and M.S. degrees from the School of Information Science and Engineering, Shandong Normal University, Jinan, China, in 2011 and 2015, respectively, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. He is currently a Postdoctoral Researcher with the Guangdong Institute of Intelligence Science and Technology, Zhuhai, China. His research interests include large-scale clustering, feature matching, multimedia, medical image processing, bioinformatics, and machine learning.
Bin Zhang received the B.S. and M.S. degrees from the School of Information Science and Engineering, Shandong Normal University, Jinan, China, in 2011 and 2015, respectively, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou, China. He is currently a Postdoctoral Researcher with the Guangdong Institute of Intelligence Science and Technology, Zhuhai, China. His research interests include large-scale clustering, feature matching, multimedia, medical image processing, bioinformatics, and machine learning.View more

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Junyu Li received the B.S. and M.S. degrees from the Guangdong University of Technology, Guangzhou, China, in 2017 and 2020, respectively. He is currently working toward the Ph.D. degree in computer science and engineering from the South China University of Technology, Guangzhou. His research interests include machine learning and image processing.
Junyu Li received the B.S. and M.S. degrees from the Guangdong University of Technology, Guangzhou, China, in 2017 and 2020, respectively. He is currently working toward the Ph.D. degree in computer science and engineering from the South China University of Technology, Guangzhou. His research interests include machine learning and image processing.View more

School of Medical Technology, Beijing Institute of Technology, Beijing, China
Bin Hu (Fellow, IEEE) was born in Beijing, China, in 1965. He received the M.S. degree in computer science from the Beijing University of Technology, Beijing, and the Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, Beijing. From 2007 to 2009, he was a Reader with the Head of Context Aware Computing Research Group, School of CTN, Birmingham City University, Birmingham, U.K. Since 2009, he has been the Dean of the School of Medical Technology, Beijing Institute of Technology, Beijing. He is also a Guest Professor with ETH Zurich, Zurich, Switzerland. His research interests include pervasive computing, cognitive computing, and mental health care. Dr. Hu was the Editor of IET Communications, Cluster Computing, Wireless Communications and Mobile Computing, the Journal of Internet Technology, Security and Communication Networks (Wiley), and Brain Informatics, and an Associate Editor for some peer reviewed journals in computer science, such as IEEE Transactions on Affective Computing and IEEE Transactions on Computational Social Systems. He is also an IET fellow, an IET Fellow Assessment Panel Member (China Committee), a Member-at-Large of ACM China, the Director of Web Intelligence Consortium (China Committee), and a Board Member of International Society for Social Neuroscience (China Committee).
Bin Hu (Fellow, IEEE) was born in Beijing, China, in 1965. He received the M.S. degree in computer science from the Beijing University of Technology, Beijing, and the Ph.D. degree in computer science from the Institute of Computing Technology, Chinese Academy of Science, Beijing. From 2007 to 2009, he was a Reader with the Head of Context Aware Computing Research Group, School of CTN, Birmingham City University, Birmingham, U.K. Since 2009, he has been the Dean of the School of Medical Technology, Beijing Institute of Technology, Beijing. He is also a Guest Professor with ETH Zurich, Zurich, Switzerland. His research interests include pervasive computing, cognitive computing, and mental health care. Dr. Hu was the Editor of IET Communications, Cluster Computing, Wireless Communications and Mobile Computing, the Journal of Internet Technology, Security and Communication Networks (Wiley), and Brain Informatics, and an Associate Editor for some peer reviewed journals in computer science, such as IEEE Transactions on Affective Computing and IEEE Transactions on Computational Social Systems. He is also an IET fellow, an IET Fellow Assessment Panel Member (China Committee), a Member-at-Large of ACM China, the Director of Web Intelligence Consortium (China Committee), and a Board Member of International Society for Social Neuroscience (China Committee).View more

School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
Jiazhou Chen (Member, IEEE) received the B.S. degree in computer science and technology from Jiaying University, Meizhou, China, the M.S. degree in software engineering from the Guangdong University of Technology, Guangzhou, China, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou. His research interests include bioinformatics, machine learning, computer vision, and image processing.
Jiazhou Chen (Member, IEEE) received the B.S. degree in computer science and technology from Jiaying University, Meizhou, China, the M.S. degree in software engineering from the Guangdong University of Technology, Guangzhou, China, and the Ph.D. degree from the School of Computer Science and Engineering, South China University of Technology, Guangzhou. His research interests include bioinformatics, machine learning, computer vision, and image processing.View more