Multi Sensor Performance Driven Data Fusion for Sign Language Synthesis

In this paper, we propose a multi sensor framework fusing hand and body motion for sign language synthesis. Microsoft Kinect and the VMG30 data gloves are used in our framework to capture finger and corresponding arm motions from two different views during gesturing. The proposed method uses a twofold approach to first coupling both hand and arm motion together for recognition, and secondly, applying boosted hierarchical Hidden Markov Model to synthesize animation.