Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation

Neural text generation has been a challenging task, among which the text representation and the beam search are crucial techniques. By improving these techniques, we propose a novel model to generate texts of higher quality in this paper. First, we leverage the global and local contextual features by combining the structure of both the recurrent neural network (RNN) and convolutional neural network (CNN) to learn a joint representation for the source text. Next, we introduce a modified diverse beam search to foster the diversity in the generated sentences during decoding, and then we rank these sentences according to its saliency score which measures the co-occurrence of keyphrases with the source text. Such a ranking mechanism promotes the semantical relevance between the source text and the generated sentence. To evaluate our model, we conduct extensive experiments on two neural generation tasks, including document summarization and headline generation. The results on both tasks show that our proposed model contributes to promising improvement in performance compared with the state-of-the-art baselines.