Visual Question Answering for Monas Tourism Object using Deep Learning

Visual Question Answering (VQA) is a machine learning task, given a pair of image and natural language visual question about the image, the task is to answer the question. It is known that there is no public VQA dataset currently available in Bahasa Indonesia. This research compiles a Monas VQA dataset that uses Bahasa Indonesia in the question and Monas, a memorial monument for Indonesian, as the image specific context to resolve the problem. This research also proposes methods to solve VQA using CNN for image embedding, techniques from the NLP field for sentence embedding e.g. Bag-of-Words, fastText, BERT, and BiLSTM, lastly multimodal machine learning to let both embedded information to interact with each other. The best performing model achieves 68.39% accuracy with architecture impact analysis presented.