Dynamically Context-sensitive Time-decay Attention for Dialogue Modeling

Spoken language understanding (SLU) is an essential component in conversational systems. Considering that contexts provide informative cues for better understanding, history can be leveraged for contextual SLU. However, most prior work only paid attention to the related history utterances and ignored the temporal information. In dialogues, prior work considers an inflexible decaying time-aware attention to allow the model to pay more attention to the most recent utterances than the least recent ones. To improve the flexibility of function design, this paper allows the model to automatically learn a time-decay attention function where the attentional weights can be dynamically decided based on the content of each role's contexts, which effectively integrates both content-aware and time-aware perspectives and demonstrates remarkable flexibility to complex dialogue contexts. The experiments on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset show that the proposed dynamically context-sensitive time-decay attention mechanisms significantly improve the state-of-the-art model for contextual understanding performance
¹
.