Semantic Similarity-based Visual Reasoning without Language Information

In this research, we propose new training data for the visual reasoning task based on semantic similarity and proposed a deep learning model that utilizes the data. The first contribution of this study is the construction of training data. Based on a total of 40 object attributes, we created a visual inference problem using only image data. As a result, a total of 6,000 datasets were built to create training and test data. We also propose a visual inference model as the second contribution of this work. The inference model shown in this study was evaluated for two tasks using ResNet50 and Vision Transformer, respectively. Based on the experimental evaluation results, we investigated the suitable pre-trained model for both single-choice binary reasoning and multiple-selection reasoning, respectively.