Composite Evolutionary GAN for Natural Language Generation with Temper Control

Natural Language Generation (NLG) is a fundamental problem in NLP. The basic approach for NLG is Maximizing the Likelihood Estimation (MLE). However, MLE is blamed for exposure bias. To tackle with the problem, Generative Adversarial Networks (GAN) based methods are proposed, which use the adversarial structure to overcome the problem. Although the reinforcement learning framework in GAN makes NLG available, non-eliminated problems such as sparse reward and intensive computations still exist. Gumble-Softmax relaxation is an alternative approach to the reinforcement learning framework to deal with discrete data. The dynamic temperature of relaxation is essential to control the model preference between quality and diversity. In this paper, we propose CEGAN, which uses an evolutionary strategy to help the generators finding the most suitable temperature during training. Besides, CEGAN uses composite discriminators to make full use of samples and embedding from generator, which is able to provide more informational signals back to generator. We conduct extensive experiments to verify the superiority of our model. The results show that CEGAN achieves higher performance compared with baselines and makes a good trade-off between quality and diversity.