Innovative technologies for the creative industries: Advanced human-machine interfaces for dynamic performance effects

Technical advances in performing arts and TV production have focused on technologies like sound, special effects, projection and the like. New developments in computer vision have made gesture and facial expression recognition possible. What is missing is the means to link these new technologies and place them under direct control of the performer. Furthermore, performers have limited capacity to interpret and respond to their audience. This research aims to develop new gesture and facial expression recognition technologies to make possible completely new forms of performance that enable performers to directly influence the physical environment and respond to audience reaction. In this article, we focus on a gesture recognition system developed for this purpose and present a novel approach for gesture recognition. We discuss the gesture modelling technique used and its main features. Experiments show accuracy of over 98.71% making the gesture recognizer suitable for our innovative stage interface.