Deep Learning Architectures for Multimodal Sentiment Analysis

In recent years, multimodal sentiment analysis has become a popular domain of research due to the rise of deep learning architectures designed to handle multiple feeding modalities such as text and images. This work delves into sentiment analysis utilizing images and text, extracted with Optical Character Recognition (OCR) from the Memotion dataset. We evaluate the performance of four different model architectures: DEIT+SBERT, DistillBERT+ResNet, RoBERTa+ResNet, and BiLSTM+ResNet. In our study, we assess these models using accuracy, F1-score, precision, and recall. According to the results of experiments, DistillBERT+ResNet achieves the best performance of 78.83% accuracy relative to others. We draw on this paper to outline how we approached our methodologies, data preprocessing steps, and comparison of model performances