A Contrastive Framework for Improving Multi-turn Response Selection

In retrieval-based dialogue systems, multi-round dialogue response selection is a major task of research and has received a lot of attention from academia and industry. BERTbased Dense Vector Retrieval employs an interactive encoder architecture to learn representations of dialogue context and candidate responses for semantic matching. However, in training session, the interaction between the context and the candidate responses exists only on the loss calculation phase. In the inference session, the prediction of the relevance score between the context and each candidate response is independent, and there is no interaction between the responses. At the same time, the context involves too many conversation rounds, resulting in the model’s inability to capture the relationship between the context and the candidate responses at a fine-grained level. To address the above issues, we propose a fine-grained pre-training and fine-tuning framework with a list-aware encoder overlaid on top of the interactive encoder. We evaluated our model on public datasets from three different domains. With our proposed framework, MRR has an average improvement of 1.1% and the average improvement of R
10
© 1, R
10
© 2 and R
10
© 5 increased by 3.4%, 2.9% and 1.0% respectively, showing the effectiveness of this approach.