Extraction of 3D hand shape and posture from image sequences for sign language recognition

We propose a novel method for extracting natural hand parameters from monocular image sequences. The purpose is to improve a vision-based sign language recognition system by providing detail information about the finger constellation and the 3D hand posture. Therefore, the hand is modelled by a set of 2D appearance models, each representing a limited variation range of 3D hand shape and posture. The single models are linked to each other according to the natural neighbourhood of the corresponding hand status. During an image sequence, necessary model transitions are executed towards one of the current neighbour models. The natural hand parameters are calculated from the shape and texture parameters of the current model, using a relation estimated by linear regression. The method is robust against large differences between subsequent frames and also against poor image quality. It can be implemented in real-time and offers good properties to handle occlusion and partly missing image information.