Efficient Deep Learning Model for Text Classification Based on Recurrent and Convolutional Layers

Natural Language Processing (NLP) systems conventionally treat words as distinct atomic symbols. The model can leverage small amounts of information regarding the relationship between the individual symbols. Today when it comes to texts; one common technique to extract fixed-length features is bag-of-words. Despite its popularity the bag-of-words feature has two major weaknesses: it ignores semantics of the words and the order of words. In this paper, we propose a neural language model that relies on Convolutional Neural Network (CNN) and Bidirectional Recurrent Neural Network (BRNN) over pre-trained word vectors. We utilize bidirectional layers as a substitute of pooling layers in CNN in order to reduce the loss of detailed local information, and to capture long-term dependencies across input sequences. We validate the proposed model on two benchmark sentiment analysis datasets, Stanford Large Movie Review (IMDB), and Stanford Sentiment Treebank (SSTb). Our model achieves a competitive advantage compared with neural language models on the sentiment analysis datasets.