(GRAVITY) Graph-Based Reasoning With Attention and Visual Information Using Transformers for Yielding Answers

This paper introduces a novel framework for Visual Question Answering (VQA) that combines Graph Attention Networks (GATs) with Transformers to improve visual-semantic reasoning. The proposed method constructs question-conditioned visual graphs to capture fine-grained relationships between image regions, enabling accurate property extraction such as color, shape, and quantity. These structured representations are integrated with natural language questions through multi-head self-attention mechanisms within a Transformer-based reasoning module, enhancing contextual understanding and answer generation. The framework is evaluated on two widely-used VQA benchmarks: GQA and VQA 2.0. On GQA, the model achieves 60.2% accuracy in the zero-shot setting and 63.4% in the few-shot setting. On VQA 2.0, it achieves an overall accuracy of 71.4%, with strong performance in reasoning-heavy categories such as “Number” and “Color.” These results demonstrate the effectiveness of our graph-enhanced attention mechanism in improving response precision and robustness across both structured and naturalistic visual question answering tasks.