We describe a real-time computer vision application that creates sound from virtual musical instruments by tracking hand and finger movements with a color-based Sequential Monte Carlo algorithm. We show how the combination of real-time hand tracking, virtual MIDI SoundFont technology, and computer graphics, can be used for producing powerful yet inexpensive augmented reality environments.
Published in:
Information Systems and Technologies (CISTI), 2011 6th Iberian Conference on
Date of Conference: 15-18 June 2011