Gesture Interaction for Small Handheld Devices to Support Multimedia Applications

Accelerometer-based gesture control is proposed as a complementary interaction modality for small handheld devices to enable a variety of multimedia applications. The motivation for experimenting with gesture interaction is justified by the personal and public domain prototype applications developed. The challenges related to developing user-dependent and independent gesture control are presented. In this article, we experiment with methods for user-dependent gesture recognition with a low number of training repetitions, and for feasible user-independent gesture recognition from a moderately large set of gestures. The user-dependent gesture recognition performance of the continuous Hidden Markov Model (HMM) is better when compared to discrete HMM with three gesture repetitions in a training set. With continuous HMM, a recognition accuracy level of 95% is obtained with or without tilt normalization, while for discrete HMM a best recognition accuracy of 90% is obtained. The user-independent gesture recognition performance with continuous HMM of 89% is considerably better compared to tests with discrete HMM, when both are obtained with cross-validation from 2,520 gestures. An important result is that the effect of using tilt normalization notably increases the user-independent gesture recognition performance by 10-15% depending on the method used. The chosen methods show great potential for gesture-based interaction in multimedia applications.