A Deep Learning-Based Bengali Visual Commonsense Reasoning System

Visual commonsense reasoning, an integral aspect of human intelligence, extends beyond mere object identification, encompassing the nuanced inference of actions, intentions, and emotions from visual content. While humans effortlessly excel in this cognitive domain, contemporary Bengali vision systems encounter obstacles in attaining comparable proficiency. This is primarily attributed to the need for advanced cognitive faculties and common-sense reasoning abilities. This research paper introduces the Bengali Visual Commonsense Reasoning System (BVCRS), differentiating from prevailing Bengali vision models that predominantly focus on tasks such as image captioning, object recognition, segmentation, and visual question answering. The Bengali Visual Commonsense Reasoning(BVCR) model, coupled with our Bangla-VCR dataset, stands as a pivotal asset for advancing research in Bengali visual understanding and reasoning. Our BVCR model aspires to capture real-world dynamics and human intentions in Bengali by incor-porating state-of-the-art deep learning techniques.We leverage a pre-trained ResNet50 for visual features and fine-tuned M-BERT for Bengali text.To extract relationships between textual and visual features for deducing answers and reasoning, a bidirectional LSTM with attention mechanism is employed. In experiments, BVCR achieved 42.6% accuracy in visual question answering and 44.2% in visual commonsense reasoning. Our work highlights the challenges in attaining human-level performance in Bengali visual reasoning, providing insights for future research.