A Live Storytelling Virtual Reality System with Programmable Cartoon-Style Emotion Embodiment

Virtual reality (VR) is a promising new medium for immersive storytelling. While previous research works on VR narrative have tried to engage audiences through nice scenes and interactivity, the emerging live streaming shows the role of a presenter, especially the conveyance of emotion, for promoting audience involvement and enjoyment. In this paper, to lower the requirement of emotion embodiment, we borrow experience from cartoon animation and comics, and propose a novel cartoon-style hybrid emotion embodiment model to increase a storyteller's presence during live performance, which contains an avatar with six basic emotions and auxiliary multimodal display to enhance emotion expressing. We further design and implement a system to teleoperate the embodiment model in VR for live storytelling. In particular, 1) we design a novel visual programming tool that allows users to customize emotional effects based on the emotion embodiment model; 2) we design a novel face tracking module to map presenters' emotional states to the avatar in VR. Our lightweight web-based implementation also makes the application very easy to use. We conduct two preliminary qualitative studies to explore the potential of the hybrid model and the storytelling system, including interviews with three experts and a workshop study with local secondary school students. Results show the potential of the VR storytelling system for education.