Abstract:
Graph Neural Networks have attracted increasing attention in recent years. However, existing GNN frameworks are deployed based upon simple graphs, which limits their appl...Show MoreMetadata
Abstract:
Graph Neural Networks have attracted increasing attention in recent years. However, existing GNN frameworks are deployed based upon simple graphs, which limits their applications in dealing with complex data correlation of multi-modal/multi-type data in practice. A few hypergraph-based methods have recently been proposed to address the problem of multi-modal/multi-type data correlation by directly concatenating the hypergraphs constructed from each single individual modality/type, which is difficult to learn an adaptive weight for each modality/type. In this paper, we extend the original conference version HGNN, and introduce a general high-order multi-modal/multi-type data correlation modeling framework called HGNN^+ to learn an optimal representation in a single hypergraph based framework. It is achieved by bridging multi-modal/multi-type data and hyperedge with hyperedge groups. Specifically, in our method, hyperedge groups are first constructed to represent latent high-order correlations in each specific modality/type with explicit or implicit graph structures. An adaptive hyperedge group fusion strategy is then used to effectively fuse the correlations from different modalities/types in a unified hypergraph. After that a new hypergraph convolution scheme performed in spatial domain is used to learn a general data representation for various tasks. We have evaluated this framework on several popular datasets and compared it with recent state-of-the-art methods. The comprehensive evaluations indicate that the proposed HGNN^+ framework can consistently outperform existing methods with a significant margin, especially when modeling implicit data correlations. We also release a toolbox called THU-DeepHypergraph for the proposed framework, which can be used for various of applications, such as data classification, retrieval and recommendation.
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 45, Issue: 3, 01 March 2023)
Funding Agency:

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Yue Gao (Senior Member, IEEE) received the BS degree from the Harbin Institute of Technology, Harbin, China, and the ME and PhD degrees from Tsinghua University, Beijing, China, where he is currently an associate professor with the School of Software.
Yue Gao (Senior Member, IEEE) received the BS degree from the Harbin Institute of Technology, Harbin, China, and the ME and PhD degrees from Tsinghua University, Beijing, China, where he is currently an associate professor with the School of Software.View more

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Yifan Feng received the BE degree in computer science and technology from Xidian University, Xi’an, China, in 2018, and the MS degree from Xiamen University, Xiamen, China, in 2021. He is currently working toward the PhD degree with the School of Software, Tsinghua University, Beijing, China.
Yifan Feng received the BE degree in computer science and technology from Xidian University, Xi’an, China, in 2018, and the MS degree from Xiamen University, Xiamen, China, in 2021. He is currently working toward the PhD degree with the School of Software, Tsinghua University, Beijing, China.View more

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Shuyi Ji received the BE degree from the School of Software, Tsinghua University, Beijing, China, in 2019, where she is currently working toward the PhD degree in machine learning.
Shuyi Ji received the BE degree from the School of Software, Tsinghua University, Beijing, China, in 2019, where she is currently working toward the PhD degree in machine learning.View more

Media Analytics and Computing Laboratory, Department of Artificial Intelligence, School of Informatics, Institute of Artificial Intelligence, Fujian Engineering Research Center of Trusted Artificial Intelligence Analysis and Application, Xiamen University, Xiamen, Fujian, China
Rongrong Ji (Senior Member, IEEE) is currently a professor and the director with the Intelligent Multimedia Technology Laboratory, School of Informatics, Xiamen University, Xiamen, China. His research interests include innovative technologies for multimedia signal processing, computer vision, and pattern recognition, with more than 100 papers published in international journals and conferences. He serves as an associate/g...Show More
Rongrong Ji (Senior Member, IEEE) is currently a professor and the director with the Intelligent Multimedia Technology Laboratory, School of Informatics, Xiamen University, Xiamen, China. His research interests include innovative technologies for multimedia signal processing, computer vision, and pattern recognition, with more than 100 papers published in international journals and conferences. He serves as an associate/g...View more

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Yue Gao (Senior Member, IEEE) received the BS degree from the Harbin Institute of Technology, Harbin, China, and the ME and PhD degrees from Tsinghua University, Beijing, China, where he is currently an associate professor with the School of Software.
Yue Gao (Senior Member, IEEE) received the BS degree from the Harbin Institute of Technology, Harbin, China, and the ME and PhD degrees from Tsinghua University, Beijing, China, where he is currently an associate professor with the School of Software.View more

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Yifan Feng received the BE degree in computer science and technology from Xidian University, Xi’an, China, in 2018, and the MS degree from Xiamen University, Xiamen, China, in 2021. He is currently working toward the PhD degree with the School of Software, Tsinghua University, Beijing, China.
Yifan Feng received the BE degree in computer science and technology from Xidian University, Xi’an, China, in 2018, and the MS degree from Xiamen University, Xiamen, China, in 2021. He is currently working toward the PhD degree with the School of Software, Tsinghua University, Beijing, China.View more

BNRist, KLISS, School of Software, BLBCI, THUIBCS, Tsinghua University, Beijing, China
Shuyi Ji received the BE degree from the School of Software, Tsinghua University, Beijing, China, in 2019, where she is currently working toward the PhD degree in machine learning.
Shuyi Ji received the BE degree from the School of Software, Tsinghua University, Beijing, China, in 2019, where she is currently working toward the PhD degree in machine learning.View more

Media Analytics and Computing Laboratory, Department of Artificial Intelligence, School of Informatics, Institute of Artificial Intelligence, Fujian Engineering Research Center of Trusted Artificial Intelligence Analysis and Application, Xiamen University, Xiamen, Fujian, China
Rongrong Ji (Senior Member, IEEE) is currently a professor and the director with the Intelligent Multimedia Technology Laboratory, School of Informatics, Xiamen University, Xiamen, China. His research interests include innovative technologies for multimedia signal processing, computer vision, and pattern recognition, with more than 100 papers published in international journals and conferences. He serves as an associate/guest editor for international journals and magazines, such as Neurocomputing, Signal Processing, and Multimedia Systems.
Rongrong Ji (Senior Member, IEEE) is currently a professor and the director with the Intelligent Multimedia Technology Laboratory, School of Informatics, Xiamen University, Xiamen, China. His research interests include innovative technologies for multimedia signal processing, computer vision, and pattern recognition, with more than 100 papers published in international journals and conferences. He serves as an associate/guest editor for international journals and magazines, such as Neurocomputing, Signal Processing, and Multimedia Systems.View more