Efficient Graph Representation with Anchor-Graph Transformer

To alleviate the local receptive issue of Graph Convolutional Network (GCN), Transformers have been exploited to capture the long-range dependence of nodes for graph data representation and learning. However, existing graph Transformers generally employ a regular self-attention module for all node-to-node message passing which needs to learn the affinities/relationships between all node’s pairs, leading to high computational cost issue. Also, they are usually sensitive to graph noises. To overcome this issue, we propose a novel graph Transformer architecture, termed AGFormer, by leveraging an anchor graph model. To be specific, AGFormer first obtains some representative anchors and then converts node-to-node message passing into anchor-to-anchor and anchor-to-node message passing processes. Thus, AGFormer performs much more efficiently and also robustly than regular node-to-node Transformers. Extensive experiments on several benchmark datasets demonstrate the benefits of the proposed AGFormer. Specifically, when the number of graph nodes reaches 15,000, AGFormer achieves a training speed that is three times faster than that of GraphTrans. Furthermore, AGFormers perform more robustly on the noised NCI109 dataset compared to GraphTrans.