Research on Automatic Generation of Social Short Text Based on Backtracking Pattern

Aiming at the problems of cyclic repetition and incoherence in text generated by the pre-trained language model based on Transformer, the backtracking pattern of phrase joint sentences is proposed to improve the generation performance of the model, by imitating the deletion and modification in the process of human writing. In the text generation process, phrase level and sentence level backtracking processes are added to control generation quality globally. The phrase level captures generation errors and tracks them through the 3–4 sliding window algorithm, controls word root selection at the word or phrase level, and improves text quality at the word level, the quality evaluator based on BERT is introduced into sentence level, which comprehensively judges the quality from fluency, consistency, redundancy and semantic correlation between sentences, and the evaluator score is used as the condition for backtracking resampling, so errors in the generation process can be detected and corrected. This method focuses on generation and adjustment, does not affect the training amount and generation mechanism of the model, and has certain universality. Based on the typical open language model GPT2 and the conditional language model PPLM, this paper realizes the high-quality generation of microblog text. Experiments show that after introducing backtracking, the model can towards expected quality direction without damaging the attribute consistency of the conditional model, and the text is more coherent and readable, which is close to human writing style. It provides a solution to improve the quality of social text generation.