Feature Correlation Aggregation: on the Path to Better Graph Neural Networks

Prior to the introduction of Graph Neural Networks (GNNs), modelling and analyzing irregular data, particularly graphs, was thought to be the Achilles’ heel of deep learning. The core concept of GNNs is to find a representation by recursively aggregating the representations of a central node and those of its neighbor, and many GNNs’ designs have demonstrated its success. However, most only focus on using the first-order information between a node and its neighbors. In this paper, we introduce a central node permutation variant function through a frustratingly simple and innocent-looking modification to the core operation of a GNN, namely the Feature cOrrelation aGgregation (FOG) module, which learns the second-order information from feature correlation between a node and its neighbors in the pipeline. By adding FOG into existing variants of GNNs, we empirically verify this second-order information complements the features generated by original GNNs across a broad set of benchmarks. A tangible boost in the model’s performance is observed where the model surpasses previous state-of-the-art results by a significant margin while employing fewer parameters. e.g., 26.202% improvement on a real-world molecular dataset using graph convolutional networks.