Research on Pre-training Model of Natural Language Processing Based on Recurrent Neural Network

In NLP(Natural Language Processing), the direct object of computer processing is not the actual natural language, but its computational model. Therefore, to truly understand the problems of NLP and find out the solutions, we need to discuss these problems from the perspective of language processing modeling. In this paper, the characteristics of NLP are analyzed, and the language model, word segmentation, part-of-speech tagging and named entity recognition in NLP are studied by using the method based on RNN(Recursive Neural Network). A dependency parsing model based on LSTM is proposed. The model is based on the feedforward neural network model mentioned above, and it is used as a feature extractor. In this paper, Minibatch method is used to optimize the training process of Embedding RNN, and this network is successfully applied to the construction of NLP pre-training model. Experimental results show that compared with baseline method, the performance of the model is greatly improved.