Deep Learning Approaches on Multimodal Sentiment Analysis

Sentiment analysis aims to uncover people's sentiment based on some information about them, often using machine learning or deep learning algorithm to determine. The importance of such a technique heavily grows because it can help companies better understand users' attitudes toward things and decide future plans. Considering the maturity of sentiment analysis on a single modal such as text and image and the advancement of social media, which allow users to post with text and image simultaneously, novel methods are designed for multimodal sentiment analysis. Because of the infancy of this field, new methods are sometimes inefficient, unstable and lack robustness and still need further study. One big challenge on the multimodal sentiment analysis is to find a proper way that can overcome heterogeneous gap between different modalities, which implies that researchers should develop a method of combining various modalities properly to get higher efficiency and accuracy. Although much research has been performed and many new models are implemented, related summary is still lacking. This paper mainly introduces four interesting and relatively novel methods of solving the links among diverse modalities. First it briefly introduces some basic definitions and terminologies in sentiment analysis, followed by several popular and common datasets on multimodal sentiment analysis. After that it gives some evaluation index to assess one model's competence. Then it shows four models and compares pros and cons among them, showing the potential trend of future study in multimodal sentiment analysis. Finally, it proposes future challenges and opportunities.