Multi-level Relational Reasoning and Attention Network for Medical Visual Question Answering

Medical Visual Question Answering (Med-VQA) aims to address clinical questions using medical radiological images. However, existing studies have mainly focused on in-putting visual and textual features into attention-based network structures (such as Transformer), neglecting the more advanced relational features present in radiological images. Therefore, we propose a Med- Vqamodel based on attention and visual relational reasoning. Firstly, we introduce a Bidirectional-guided Attention Module, enabling the model not only to use questions to guide attention to important regions in the image, but also to utilize the image to guide attention to key vocabulary in the question. Secondly, we design a Multi-level Visual Relational Module that models both global and local visual features. We employ a graph convolutional neural network to extract latent visual relational features, enriching the semantic information con-tained in visual features. Experimental results on the VQA-RAD dataset and SLAKE dataset show that our model outperforms other state-of-the-art Med- Vqamodels with an overall accuracy of 77.2 % and 80.3 %, respectively, while guaranteeing the model performance.