Virtual Stage: Interactive Puppeteering in Mixed Reality

This interdisciplinary paper presents an application of a virtual studio combined with interactive character animation, using standard, consumer grade technology. Virtual reality setups and depth-based cameras are now available for a broad audience. We show that these devices support the realization of a low-cost virtual studio setup with free camera movement and person tracking. We combine this with interactive character animation using natural user interfaces. Our proposed method to control the characters was inspired by animation and theater puppetry. The degrees-of-freedom to perform movements are limited, but easy to learn and powerful enough to enable for expressive performances in a cartoon-style environment. The proposed setup has great potential for the swift production of interactive live screen shows.