Recognition of complicated gesture in real-time interactive system

This paper presents a novel gesture recognition method for a real time interactive system. The feature of this method is that both 'what kind of gesture is performed' and 'how the gesture is performed' are recognized by comparing a model gesture with an input one in a 'gesture space'. The space is a low-dimensional space constituted by the Karhunen-Loeve expansion of model image sequences, which represents the human part to make a gesture. To extract an interesting region of input image robustly in real time, we developed the masked template model which is an improved template matching method for gesture recognition. Using our recognition method, we describe a real time interactive system, called the Virtual Conductor System, which can control the music played by a computer using the gesture recognition results.