A Deep Q-Network Based Reward Shaping for Task-Oriented Dialogue Agent

The dialogue agent plays a core role in the pipeline of a task-oriented dialogue system, selecting dialogue actions for dialogue generation and determining overall dialogue performance. A promising approach for dialogue agent learning is deep reinforcement learning. However, the acquisition of reward signals occurs only after the dialogue is complete, leading to delayed rewards. In this paper, we design a dynamic reward function based on various dialogue trajectories and lengths. This function rewards conversations based on the dialogue state and the position of the turns. We incorporate the designed reward function into the deep Dyna-Q reinforcement learning framework to model dialogue agents. Experiments conducted on a movie-ticket booking dataset demonstrate that our proposed reward shaping methods achieve optimal results compared to baseline dialogue agents. The experiments validate the effectiveness of the proposed reward shaping model.