GCN-LRP explanation: exploring latent attention of graph convolutional networks

Graph convolutional networks (GCNs) have been successfully applied to many graph data on various learning tasks such as node classification. However, there is limited understanding of the internal logic and decision patterns of GCNs. In this paper, we propose a layer-wise relevance propagation based explanation method for GCNs, namely GCN-LRP, to explore the latent pattern of GCNs. Then, we use three well-known citation network data sets and synthetic graph data sets for node classification tasks with GCN-LRP explanation, and experimentally identify latent attentions when GCNs aggregates information from the node and its neighboring nodes: (i) GCNs pay more attention to the classified node when comparing with its neighbors; (ii) GCNs do not pay attention to all the neighboring nodes equally, and a few neighboring nodes received more attention than others. Moreover, we further theoretically analyze and find that: (i) the latent attentions come from the recursively aggregating of GCNs; (ii) the neighboring nodes, which share enough neighbors with classified node, would receive more attention than other neighbors; (iii) the latent attention could hardly be changed by model training. We also discuss the advantage and limitations of GCNs introduced by the latent attentions, and implications of our findings for graph data learning with GCNs.

I. Introduction

Tackling learning tasks on graph-structured data usually need extracting representations from nodes' features and graph structure. For this purpose, graph neural networks (GNNs) [1]-[8] compute new representations for each node by aggregating its neighbors' feature vectors or representations. Inspired by convolutional neural networks (CNNs) successfully used in computer vision [9]-[10], researchers began to extend convolution operation onto graphs as an aggregation method. Most works were based on spectral graph convolutional neural networks (GCNNs) [11]-[12], and graph convolutional networks (GCNs) was introduced in [3] which applied some simplifications to the basic frameworks and improved model's scalability and classification performance successfully. GCNs have been improved and applied to various application scenes [13]-[14].