The Impact of Multimodal Fusion methods and Debiasing Techniques for Medical VQA

Medical Visual Question Answering (Med-VQA) combines computer vision (CV) and natural language processing (NLP) to help make clinical decisions and medical education by answering questions about medical images. However, developing powerful Med-VQA models faces several issues, including the complex nature of medical images, the availability of labeled data, and dataset biases. This research examines two critical points of Med-VQA: (1) multimodal fusion techniques for combining visual and textual information, and (2) the effectiveness of unbiased learning approaches using data augmentation and class weighting methods. The paper evaluates three fusion approaches (concatenation-based, attention-based, and bilinear attention net- works) on two benchmark datasets: SLAKE and PathVQA. Our experiments show that attention-based approaches perform well, with an accuracy of 58.47% on PathVQA. The study of unbiased learning techniques shows that the adopted class weighting method does not improve Med-VQA performance as expected, while using the image augmentation technique alone outperforms combinations with the class weighting technique.