Generating Style-Specific Chinese Tang Poetry With a Simple Actor-Critic Model

Recent studies in sequence-to-sequence learning demonstrate that recurrent neural network (RNN) encoder-decoder structure can do well in Chinese classical poetry generation. With topic words or first line as the encoder input, an entire poem is then incrementally generated by the decoder from left to right with the highest probability. However, this locally incremental nature of decoding model can lead to the incongruity of style between the front and back of the poem generated. Inspired by the behavior of people tending to plan and associate the following parts in advance when they are writing poems, this paper employs a simple actor-critic method to generate style-specific Chinese poems. We design a style matching reward function and employ a value network with Monte Carlo search as the critic to estimate the future rewards of the desired style for poem generation. This approach makes the generation process more flexible and controllable. The experimental results demonstrate that our approach can generate three specific styles of high-quality poetry, and enhance the consistency of style of generated poems.