Transfer Pretrained Sentence Encoder to Sentiment Classification

Deep learning has been demonstrated to be very effective in many difficult tasks of computer vision(CV) and natural language processing(NLP). But this usually depends on a large number of available training samples to learn, and powerful computing resources (like GPU). In the tasks of computer vision, multiple deep layers of the model are initialized with weights pre-trained on the large-scale datasets like ImageNet, which improves the training speed of the network but does not reduce the performance of the network. Inspired by this, in this paper, in order to demonstrate that transfer is also effective in natural language processing(NLP), we transfer the encoder parts of two models trained under different tasks of NLP to sentiment classification. In the result, we can find that even if we use a simple classifier, we still get good accuracy. And we propose a persuasive explanation for the phenomenon that why the similarity of the representation vectors obtained by different encoders under the same sentence is not high.