Empowering Dialogue Systems with Affective and Adaptive Interaction: Integrating Social Intelligence

As spoken language technologies continue to rapidly advance, the need for more natural human-computer interaction in our daily lives becomes increasingly apparent to leverage the full potential of Spoken Dialogue Systems (SDSs). In this paper, we identify key advances needed to empower machines with social intelligence in spoken language interaction from both perception and synthesis aspects. To illustrate the limitations of current SDSs, we outline two social intelligence capabilities that are present in human conversations but not yet fully realized in machines, namely affect anticipation and voice adaptation. We provide experimental analysis of affect anticipation and theoretical analysis of voice adaptation to demonstrate the feasibility and necessity of these capabilities for empowering dialogue systems with affective and adaptive interaction to achieve human-level social intelligence.