MoodFlow: Orchestrating Conversations with Emotionally Intelligent Avatars in Mixed Reality

MoodFlow presents a novel approach at the intersection of mixed reality and conversational artificial intelligence for emotionally intelligent avatars. Through a state machine embedded in user prompts, the system decodes emotional nuances, enabling avatars to respond authentically to the spectrum of human emotions. Our system employs expressive avatars with a shared structure, allowing for seamless animation transferability between avatars with distinct outlook. The avatars, optimized for mixed reality, incorporate low-poly designs and toon shader stylization. This immersive journey transforms virtual conversations into open-ended dialogues, where avatars go beyond scripted interactions, adapting in real-time based on emotional context. Beyond entertainment, the approach envisions diverse applications, including virtual therapy, education, entertainment, corporate communication, and social interactions by opening doors to emotionally rich experiences across sectors.