User Interface to Input Three-Dimensional Information with User's Palm

This paper describes a contact-free input method which does not use a pointing device. The proposed device has an input function to capture the user's palm motions and an output function which displays a screen directed by the palm motions. First, a miniature camera captures the palm motions and inputs the images. Next, a color conversion appropriate for palms, and wrinkle detection by thinning and detecting the optical flow are executed. Because our previous proposals only detected the optical flows of right-to-left and up-and-down movements of the palm, the system could only move a cursor displayed on the output screen. This paper extends the system to detect palm motions toward and away from the camera. This allows new functions, e.g. selecting menus, clicking icons, and zooming the viewing area in and out, based on the three-dimensional motions of the palm. To verify the proposed user interface, evaluation experiments were conducted. First, a palm-size verification system was constructed by using an embedded processor for consumer use. Next, the optical flows of palm wrinkles were detected based on the system. Thirdly, the direction and the distance of the palm motions were estimated from the detected optical flows and were related to the screen image of the output display. Usability testing showed that the optical flows of the palm wrinkles were detected normally. Therefore, the proposal has realized a new user interface for three-dimensional input without using existing pointing devices.