Improving Matching Models With Contextual Attention for Multi-Turn Response Selection in Retrieval-Based Chatbots

Multi-turn response selection is an important task in artificial intelligence. Early methods match each utterance with a response to obtain the matching information between utterance and response, then aggregate the matching vectors in chronological order. They are lightweight but ignore the dependencies between utterances, which is very important for mining useful matching information in utterance-response pair. Recently, some PLM-based methods can consider both relations between utterance and response and relations within utterances. However, they cost huge computational resource and suffer from loss of information due to the maximum length limit. In this research, we propose a lightweight, effective and low-loss method, CSMN. We initially expand the traditional attention to context-aware attention, making the model to dynamically learn complete matching information from response, utterance and context during the utterance-response matching. A hierarchical context-aware aggregation network is then applied for the further improvement of the proposed model. Experimental results on three large-scale dialogue datasets collected from social networks demonstrate the effectiveness of our proposed model. CSMN outperforms all traditional methods and is comparable to existing PLM-based methods with a extremely low cost of computational resource, which improves the response quality and user experience in multi-turn dialogue systems, and has important practical applications in resource-constrained environments.