Surrounding Keyword-Aware Self-Attention Network for Visual Question Answering

With the emergence of the Transformer architecture, self-attention networks have gained widespread attention as efficient and parallel mechanisms for capturing global information across various domains. However, traditional self-attention networks face challenges in accurately localizing key textual elements and allocating importance to individual words in complex multimodal Visual Question Answering (VQA) tasks, resulting in suboptimal performance. To address this issue, we propose a Surrounding Keyword-aware Self-Attention Network (SKASAN) designed to focus on the local contextual information surrounding key textual elements while integrating global information, thereby more effectively capturing crucial information necessary for problem-solving. Specifically, we introduce varying degrees of Gaussian deviation to achieve localized attention focusing and leverage global attention across layer information to guide stacked feature layer extraction, thereby correcting the attention distribution. Experimental results on the VQA-v2 dataset demonstrate that our proposed SKASAN significantly improves the rationality of attention distribution, leading to notable performance enhancements.