Goal-directed Sequence Generation with Simulation Feedback Method

Reinforcement Learning (RL) is used in the language sequence generation. The reason for the occurrence of neglected loss is that simulation representing the environment calculates rewards of environment without direct control rules and uses self-feedback strategies typically. This process is that the agent provides feedback to itself as an environment. To overcome drawback, we propose a goal-directed language sequence generation method for Natural Language Generation (NLG) tasks and a reliable implementation strategy. It helps the agent learn generation rules without human help and improve the accuracy of goal. The work demonstrates the way of using a preliminary generator as the environment simulator during the initialization phase to provide feedback for the training generator on behalf of the environment. When the training level meets the decision condition, the replacement strategy is selected for correction because the environment simulator is not enough to provide positive feedback. The discriminator provides the specific target reward, and the guide content is generated in the existing method based on RL and Generative Adversarial Net (GAN) method. Experimental results on the synthetic data and real-world tasks prove the advantages of goal-directed sequence generation with simulation feedback method over sequence generative adversarial nets models.