Efficient Message Passing Algorithm and Architecture Co-Design for Graph Neural Networks

Graph neural networks (GNNs) are a promising method for learning graph representations and demonstrate remarkable performance on various graph-related tasks. Existing typical GNNs exploit the neighborhood message passing scheme that subtly aggregates feature messages from neighbor nodes to update the node representations. Despite the effectiveness of this scheme, its complex computational model heavily relies on the graph structure, which hinders their scaling to realistic large-scale graph applications. Although several custom accelerators have been proposed to speed up GNNs, these hardware-specific optimization techniques fail to address the fundamental problem of high computational complexity in GNNs. To tackle this challenge, we propose a dedicated algorithm-architecture co-design framework, dubbed MePa, which aims to improve GNN execution efficiency by coordinating algorithm- and hardware-level innovations. Specifically, with an in-depth analysis of GNN message-passing algorithms and potential speedup opportunities, we first propose an efficient message-passing algorithm that can dynamically prune task-irrelevant graph data at multiple granularity, including channel/edge/node-wise. This approach significantly reduces the overall complexity of GNN with negligible accuracy loss. A novel architecture is designed to support dynamic pruning and translate it into performance improvements. Elaborate pipelines and specialized optimizations jointly improve performance and decrease energy consumption. Compared to the state-of-the-art GNN accelerator AWB-GCN, MePa achieves on average $\text{1.95} \times$ speedups and $\text{2.6} \times$ energy efficiency.