Energy-Efficient SpMM Kernels for GATs and GCNs on a CGLA

Graph Neural Networks (GNNs) have shown excellent performance in various fields, such as atomic structure estimation, chemical reaction modeling, and information retrieval aids in Large Language Models (LLMs). Among GNN models, Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs) are widely applied due to their computational efficiency and scalability. GATs provide higher inference accuracy through the Attention mechanism. Despite their advantages, these models face a significant bottleneck in Sparse Matrix-Matrix Multiplication (S $p$MM). Current SpMM platforms, which include potent CPUs, GPUs, and existing hardware accelerators, struggle to balance high performance and low power consumption, making them unsuitable for low-power graph applications. To address these challenges, we propose S $p$MM kernels that leverages the Coarse-Grained Reconfigurable Array (CGRA)-based accelera-tor IMAX3 (In-Memory Accelerator eXtension 3). Since the data flow in IMAX3 is static, padding is required to align the number of nonzero elements in each row, increasing data volume. To minimize this overhead, we introduced a new data format with minimal padding and optimized instruction mapping for seamless operations on IMAX3. We estimated the performance when converted to an ASIC using TSMC's 28nm process libraries. The results show that IMAX3 achieves a significant improvement in energy-delay product (EDP), outperforming the CPU i9-10940X by factors ranging from 20.24 x to 1,334.12x, the GPU RTX3090 by up to 1.54x, and NVIDIA Jetson AGX Orin by up to 172.31x. Compared to prior accelerator work, our kernel delivers EDP improvements ranging from 46.71x to 3,542.76x. These results highlight the exceptional efficiency and scalability of the proposed IMAX3-based SpMM kernel in overcoming GNN bottlenecks.