Abstract:
We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. While most current methods focus on explaining gr...Show MoreMetadata
Abstract:
We investigate the explainability of graph neural networks (GNNs) as a step toward elucidating their working mechanisms. While most current methods focus on explaining graph nodes, edges, or features, we argue that, as the inherent functional mechanism of GNNs, message flows are more natural for performing explainability. To this end, we propose a novel method here, known as FlowX, to explain GNNs by identifying important message flows. To quantify the importance of flows, we propose to follow the philosophy of Shapley values from cooperative game theory. To tackle the complexity of computing all coalitions’ marginal contributions, we propose a flow sampling scheme to compute Shapley value approximations as initial assessments of further training. We then propose an information-controlled learning algorithm to train flow scores toward diverse explanation targets: necessary or sufficient explanations. Experimental studies on both synthetic and real-world datasets demonstrate that our proposed FlowX and its variants lead to improved explainability of GNNs.
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 46, Issue: 7, July 2024)

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Shurui Gui received the BS degree in computer science from the University of Science and Technology of China, in 2020. He is currently working toward the PhD degree with Texas A&M University, College Station, Texas. His research interests include deep learning, explainability, OOD generalization, and causality.
Shurui Gui received the BS degree in computer science from the University of Science and Technology of China, in 2020. He is currently working toward the PhD degree with Texas A&M University, College Station, Texas. His research interests include deep learning, explainability, OOD generalization, and causality.View more

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Hao Yuan received the BS degree in computer science from the University of Science and Technology of China, in 2012, the MS degree in computer science from the Memorial University of Newfoundland, in 2016, and the PhD degree in computer science from Texas A&M University, College Station, Texas, in 2021. He is currently a research scientist with Meta Platforms, Inc. His research interests include machine learning, deep lea...Show More
Hao Yuan received the BS degree in computer science from the University of Science and Technology of China, in 2012, the MS degree in computer science from the Memorial University of Newfoundland, in 2016, and the PhD degree in computer science from Texas A&M University, College Station, Texas, in 2021. He is currently a research scientist with Meta Platforms, Inc. His research interests include machine learning, deep lea...View more

Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, Anhui, China
Jie Wang (Senior Member, IEEE) received the BSc degree in electronic information science and technology from the University of Science and Technology of China, Hefei, China, in 2005, and the PhD degree in computational science from the Florida State University, Tallahassee, FL, in 2011. He is currently a professor with the Department of Electronic Engineering and Information Science at University of Science and Technology...Show More
Jie Wang (Senior Member, IEEE) received the BSc degree in electronic information science and technology from the University of Science and Technology of China, Hefei, China, in 2005, and the PhD degree in computational science from the Florida State University, Tallahassee, FL, in 2011. He is currently a professor with the Department of Electronic Engineering and Information Science at University of Science and Technology...View more

School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
Qicheng Lao received the BS degree in medicine from Fudan University, China, the MSc degree in experimental medicine from McGill University, Canada, and the PhD degree in computer science from Concordia University, Montreal. He is currently a post-doctoral fellow with the Montreal Institute for Learning Algorithms (MILA). His research interests include multimodal representation learning, and machine learning methods appli...Show More
Qicheng Lao received the BS degree in medicine from Fudan University, China, the MSc degree in experimental medicine from McGill University, Canada, and the PhD degree in computer science from Concordia University, Montreal. He is currently a post-doctoral fellow with the Montreal Institute for Learning Algorithms (MILA). His research interests include multimodal representation learning, and machine learning methods appli...View more

West China Biomedical Big Data Center, West China Hospital, Chengdu, Sichuan, China
Kang Li received the PhD degree in mechanical engineering from the University of Illinois at Urbana-Champaign, Champaign, IL, in 2009. He is an associate professor with the Department of Orthopaedics, New Jersey Medical School (NJMS), Rutgers University, Newark, NJ, and a graduate faculty member of the Department of Computer Science, Rutgers University. His research interests include AI in healthcare, musculoskeletal biom...Show More
Kang Li received the PhD degree in mechanical engineering from the University of Illinois at Urbana-Champaign, Champaign, IL, in 2009. He is an associate professor with the Department of Orthopaedics, New Jersey Medical School (NJMS), Rutgers University, Newark, NJ, and a graduate faculty member of the Department of Computer Science, Rutgers University. His research interests include AI in healthcare, musculoskeletal biom...View more

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Shuiwang Ji (Fellow, IEEE) received the PhD degree in computer science from Arizona State University, Tempe, Arizona, in 2010. Currently, he is a professor and presidential impact fellow with the Department of Computer Science and Engineering, Texas A&M University, College Station, Texas. His research interests include machine learning and AI for science. He received the National Science Foundation CAREER Award in 2014. H...Show More
Shuiwang Ji (Fellow, IEEE) received the PhD degree in computer science from Arizona State University, Tempe, Arizona, in 2010. Currently, he is a professor and presidential impact fellow with the Department of Computer Science and Engineering, Texas A&M University, College Station, Texas. His research interests include machine learning and AI for science. He received the National Science Foundation CAREER Award in 2014. H...View more

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Shurui Gui received the BS degree in computer science from the University of Science and Technology of China, in 2020. He is currently working toward the PhD degree with Texas A&M University, College Station, Texas. His research interests include deep learning, explainability, OOD generalization, and causality.
Shurui Gui received the BS degree in computer science from the University of Science and Technology of China, in 2020. He is currently working toward the PhD degree with Texas A&M University, College Station, Texas. His research interests include deep learning, explainability, OOD generalization, and causality.View more

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Hao Yuan received the BS degree in computer science from the University of Science and Technology of China, in 2012, the MS degree in computer science from the Memorial University of Newfoundland, in 2016, and the PhD degree in computer science from Texas A&M University, College Station, Texas, in 2021. He is currently a research scientist with Meta Platforms, Inc. His research interests include machine learning, deep learning, and explainability.
Hao Yuan received the BS degree in computer science from the University of Science and Technology of China, in 2012, the MS degree in computer science from the Memorial University of Newfoundland, in 2016, and the PhD degree in computer science from Texas A&M University, College Station, Texas, in 2021. He is currently a research scientist with Meta Platforms, Inc. His research interests include machine learning, deep learning, and explainability.View more

Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, Anhui, China
Jie Wang (Senior Member, IEEE) received the BSc degree in electronic information science and technology from the University of Science and Technology of China, Hefei, China, in 2005, and the PhD degree in computational science from the Florida State University, Tallahassee, FL, in 2011. He is currently a professor with the Department of Electronic Engineering and Information Science at University of Science and Technology of China. His research interests include reinforcement learning, knowledge graph, large-scale optimization, deep learning, etc.
Jie Wang (Senior Member, IEEE) received the BSc degree in electronic information science and technology from the University of Science and Technology of China, Hefei, China, in 2005, and the PhD degree in computational science from the Florida State University, Tallahassee, FL, in 2011. He is currently a professor with the Department of Electronic Engineering and Information Science at University of Science and Technology of China. His research interests include reinforcement learning, knowledge graph, large-scale optimization, deep learning, etc.View more

School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
Qicheng Lao received the BS degree in medicine from Fudan University, China, the MSc degree in experimental medicine from McGill University, Canada, and the PhD degree in computer science from Concordia University, Montreal. He is currently a post-doctoral fellow with the Montreal Institute for Learning Algorithms (MILA). His research interests include multimodal representation learning, and machine learning methods applied to healthcare.
Qicheng Lao received the BS degree in medicine from Fudan University, China, the MSc degree in experimental medicine from McGill University, Canada, and the PhD degree in computer science from Concordia University, Montreal. He is currently a post-doctoral fellow with the Montreal Institute for Learning Algorithms (MILA). His research interests include multimodal representation learning, and machine learning methods applied to healthcare.View more

West China Biomedical Big Data Center, West China Hospital, Chengdu, Sichuan, China
Kang Li received the PhD degree in mechanical engineering from the University of Illinois at Urbana-Champaign, Champaign, IL, in 2009. He is an associate professor with the Department of Orthopaedics, New Jersey Medical School (NJMS), Rutgers University, Newark, NJ, and a graduate faculty member of the Department of Computer Science, Rutgers University. His research interests include AI in healthcare, musculoskeletal biomechanics, medical imaging, healthcare engineering, design and biorobotics, and human factors/ergonomics.
Kang Li received the PhD degree in mechanical engineering from the University of Illinois at Urbana-Champaign, Champaign, IL, in 2009. He is an associate professor with the Department of Orthopaedics, New Jersey Medical School (NJMS), Rutgers University, Newark, NJ, and a graduate faculty member of the Department of Computer Science, Rutgers University. His research interests include AI in healthcare, musculoskeletal biomechanics, medical imaging, healthcare engineering, design and biorobotics, and human factors/ergonomics.View more

Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA
Shuiwang Ji (Fellow, IEEE) received the PhD degree in computer science from Arizona State University, Tempe, Arizona, in 2010. Currently, he is a professor and presidential impact fellow with the Department of Computer Science and Engineering, Texas A&M University, College Station, Texas. His research interests include machine learning and AI for science. He received the National Science Foundation CAREER Award in 2014. He is currently an associate editor for IEEE Transactions on Pattern Analysis and Machine Intelligence, ACM Transactions on Knowledge Discovery from Data, and ACM Computing Surveys. He regularly serves as an area chair or equivalent roles for data mining and machine learning conferences, including AAAI, ICLR, ICML, IJCAI, KDD, and NeurIPS. He is a fellow of AIMBE, and a distinguished member of ACM.
Shuiwang Ji (Fellow, IEEE) received the PhD degree in computer science from Arizona State University, Tempe, Arizona, in 2010. Currently, he is a professor and presidential impact fellow with the Department of Computer Science and Engineering, Texas A&M University, College Station, Texas. His research interests include machine learning and AI for science. He received the National Science Foundation CAREER Award in 2014. He is currently an associate editor for IEEE Transactions on Pattern Analysis and Machine Intelligence, ACM Transactions on Knowledge Discovery from Data, and ACM Computing Surveys. He regularly serves as an area chair or equivalent roles for data mining and machine learning conferences, including AAAI, ICLR, ICML, IJCAI, KDD, and NeurIPS. He is a fellow of AIMBE, and a distinguished member of ACM.View more