A hand shape recognizer from simple sketches

Hand shape recognition is one of the most important techniques used in human-computer interaction. However, it often takes developers great efforts to customize their hand shape recognizers. In this paper, we present a novel method that enables a hand shape recognizer to be built automatically from simple sketches, such as a "stick-figure" of a hand shape. We introduce the Hand Boltzmann Machine (HBM), a generative model built upon unsupervised learning, to represent the hand shape space of a binary image, and formulate the user provided sketches as an initial guidance for sampling to generate realistic hand shape samples. Such samples are then used to train a hand shape recognizer. We evaluate our method and compare it with other state-of-the-art models in three aspects, namely i) its capability of handling different sketch input, ii) its classification accuracy, and iii) its ability to handle occlusions. Experimental results demonstrate the great potential of our method in real world applications.