A response generation method of chat-bot system using input formatting and reference resolution

Recently, the demand for chatting with dialogue systems has been increasing. However, one of the significant problems of chat dialog systems is that they cannot respond appropriately to users’ spontaneous speech since systems often respond inconsistently to past topics due to their weak dialog history management. To avoid this problem, the system tends to generate conservative, simple and short speech for consistent conversation, which leads to a boring dialogue. In this study, we constructed a chat dialog system that compensates for the lack of content of input sentences by two components. The first is an attention-based Japanese Text-to-Text Transfer Transformer(T5) model fine-tuned to spoken-to-written language conversion tasks for input formatting. The second is Centering Theory for the reference resolution. Then, we used a Japanese GPT-2-based response generation model adapted to the written language domain. Input shaping allows the dialogue content before the last three utterances to be stored in pronouns and generate responses. As a result, the system could output a response with richer content than the results without using our method and, according to the 27 evaluators, was able to generate more natural dialogue than without the method.