Exploration of Lightweight Neural Network Architectures for Sentiment Analysis

The increasing computational burden associated with traditional Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) is driving the need for lightweight, efficient models, particularly for edge computing and low-power devices. This study explores the performance and computational costs of various lightweight neural network architectures, including Long Short-Term Memory (LSTM), SimpleRNN, and multiple versions of 1D CNNs, as well as a 1D ResNet and a DenseNet model. The experimental setup evaluates each model across the same problem domain, with the intention of determining the most efficient and effective architecture. Methods involved layering combinations of convolutional, pooling, normalization, dropout, and densely connected layers. Some models were augmented with advanced techniques like residual connections and learning rate decay for better performance and generalization. Results indicate that simplified 1D CNNs perform remarkably well in terms of both accuracy and training efficiency. More specifically, 1D CNN models achieved comparable or better performance metrics than their LSTM and SimpleRNN counterparts, with significantly lower computational costs. This was particularly true for CNN models with simpler architectures, emphasizing the possibility of achieving high performance with reduced complexity. In conclusion, findings provide strong evidence that simpler 1D CNN models can serve as effective and computationally inexpensive alternatives to more complex neural network architectures, potentially facilitating broader adoption in resource constrained environments.