Arabic Sentiment Analysis and Sarcasm Detection Using Probabilistic Projections-Based Variational Switch Transformer

Text classification is a common task in natural language processing (NLP), where the objective is to assign predefined categories or labels to a given text. Detecting sarcasm and classifying sentiment and dialect in NLP has practical applications, including spam detection, topic classification, and sentiment analysis. However, sarcasm and sentimental expressions, such as irony, humor, or criticism, can be difficult to identify through traditional NLP methods due to their implicit nature. To address this, we propose a Modified Switch Transformer (MST) for detecting sarcasm and classifying sentiment and dialect in Arabic text data. Our approach includes two key contributions: Variational Enmesh Expert’s Routing (
$VE_{e}R$
) and Probabilistic Projections (
$P_{\phi} $
). The switch transformer model incorporates probabilistic projections using a Variational Spatial Gated Unit-MLP to enhance the embedding generation mechanism. This updated mechanism introduces a variational aspect, providing dynamic control over the flow of information in the network, in contrast to the simpler embedding generation phase used in the original switch transformer. Moreover, we incorporate Variational Enmesh Expert’s Routing, which utilizes a hierarchical set of Variational experts, where each expert is a small and variational-directed acyclic graph network. The
$VE_{e}R$
routing technique allows the network to dynamically choose which path to take at each layer based on the input, using a set of weights learned during training to determine the best route for a given input. Instead of optimizing route paths deterministically, we utilize Variational Inference and model each route as a random variable from a distribution. Our study evaluates the effectiveness of the Modified Switch Transformer (MST) model on the ArSarcasm Dataset, which includes Arabic language data related to sarcasm, dialect, and sentiments. We compare the performance of our proposed model with existing state-of-the-art models in the literature. The results show that the switch transformer outperforms other models in detecting sarcasm and also performs well in classifying sentiment and dialect.