Enhancing Visual Question Answering: A Novel Approach to Determining Question Relevance

Visual Question Answering (VQA) strives to respond to open-ended natural language queries posed about an image. Existing systems face a notable limitation in handling absurd or irrelevant questions, often providing archetype-based answers to such queries. Recognizing and assessing the relevance of questions to an image is crucial for advancing VQA capabilities. There are two aspects to addressing this challenge: visual vs. non-visual determination and true vs. false premises. This paper focuses on proposing a solution for the latter. QPC-Sim currently stands as the state-of-the-art for True vs. False Premise, leveraging the similarity between questions, premises, and captions. The study presents a model that integrates visual concepts to enhance the existing state-of-the-art and analyze its features. The study also explores an alternative approach to question generation for the model, incorporating hidden premises within the image, thereby contributing to the evolution of Visual Question Answering with relevance to proving a better use experience.