Can GNNs Outperform GAT? A Study of Attention Efficiency for Document Classification in Graphs

The emergence of Graph Attention Networks (GATs) has led to promising results, suggesting potential advantages over traditional Graph Neural Networks (GNNs). However, while various GAT and GNN models have been used in document classification tasks, a comprehensive comparison of their relative performance is limited. This study aims to fill this gap by evaluating whether GAT models indeed surpass traditional GNNs in document classification accuracy and analyzing which GAT variants are most effective. Our extensive experimentation show that GAT models, particularly recent variants, outperform standard GNNs and the baseline GAT. We attribute this improved performance to the adaptive weighting of neighbors' information within the attention mechanism, contrasting with the fixed aggregation in traditional GNNs.