GIFTS: Efficient GCN Inference Framework on PyTorch-CPU via Exploring the Sparsity

Graph Convolutional Networks (GCNs) are gaining attraction in AI research due to their ability to learn from graph data effectively. However, deploying GCNs on CPUs presents substantial challenges, as large-scale graph structures can lead to intensive computational and memory demands. While acceleration methods dedicatedly designed for GCNs on CPU platforms have emerged, they may not be the optimal solution as they tend to overlook the opportunity gained from the sparsity in the feature and adjacency matrices, leading to unsatisfactory memory and computation savings. In this paper, we propose a progressive two-step algorithm called GIFTS to accelerate GCN inference on CPUs by making use of the dynamic sparsity in the feature matrix and the static sparsity in the adjacency matrix. The first step introduces an online adaptive compression approach that performs a selective value-level compression on the part of feature vectors that will be frequently accessed during GCN inference. To further reduce the redundant bit width in feature vectors, we employ a bit-level pruning approach to narrow down bandwidth. The second step designs an offline degree-aware scheduling approach, which aims at balancing workloads caused by the irregular sparsity in the adjacency matrix during the GCN training process, given the observation that the adjacency matrix remains unchanged during inference. This step distributes workloads based on the nodes' degree in a coarse-grained fashion, considering both workload distribution and data locality. Through comprehensive evaluation, we demonstrate that GIFTS consistently outperforms the PyTorch Geometric (PyG) and DistGNN frameworks in terms of execution time, while incurring negligible accuracy loss. The source code is available at https://github.com/ACA-Lab/GIFTS.