A Virtual Button Interface using Fingertip Movements

This paper presents a virtual push-button interface created by drawing a shape or line in the air with a fingertip. As an example, we develop a four-button interface for entering multi-digit numbers by making pushing gestures within an invisible 2 x 2 button matrix inside a square drawn by the user. Trajectories of the fingertip movements entering numbers are captured with a 3D position sensor mounted on the tip of the forefinger. Two methods are used for recognizing the gestures of drawing a square and pushing buttons. The first method recognizes the numbers according to the relative positions between the range of the drawn square and pushed points. The second method models normalized fingertip movements with hidden Markov models (HMMs). They are evaluated with four experimental participants, and 92.5% and 95.4% recognition rates are obtained, respectively.