Image-text Similarity Guided Fusion Network for Multimodal Aspect Sentiment Analysis

The purpose of multimodal aspect sentiment analysis is to classify the sentimental polarities of the mentioned targets from graphic and textual data. However, previous approaches neglected fine-grained semantic associations between images and text, as well as the associations between image targets, resulting in limitation in multimodal aspect sentiment analysis. To tackle these difficulties, we suggest a fusion network model guided by image text similarity (ISGF). The fusion network utilizes aspect guidance attention and multimodal representation fusion module based on image text similarity to obtain effective multimodal fusion information. Align and fuse multimodal features by comparing learning methods, and assist in predicting the final answer. Our experimental outcomes on two public sentiment datasets demonstrate the feasibility and effectiveness of the model ISGF.