A Hybrid Method of Self-Supervised Graph Embedding, Siamese Networks, and Transformers for Sentiment Analysis in Persian Language

Recently, Graph Representation Learning (GRL) methods such as biased random walks became popular and are under attention in deep learning literature. Recent works using graph-structured data as inputs in deep learning models show great performance and effectiveness of graph representation learning. Therefore some research has been done in this domain to transform other types of data and problems into graph-structured data to use as inputs for neural networks. In this research, we propose a self-supervised approach for Persian sentiment analysis using combined representation learning and siamese network to improve performance. By leveraging a self-supervised approach, the first features of words are extracted in our constructed graph using biased random walks. After that, the extracted feature vectors are improved by training a siamese network in a self-supervised manner while pseudo-labels are provided by graph edges. Eventually, the improved features are fed as inputs to a transformer-based neural network. The obtained results of experiments demonstrate that our model outperforms the state-of-the-art models of sentiment analysis tasks in terms of classification accuracy over two Persian user comment datasets. An ablative study is conducted for justifying the effectiveness of our proposed components and the achieved results confirm the efficiency of our proposed methods.