Vision-based realtime human hand direction recognition system using affine invariants

A vision-based human interface system which enables a user to move a target object in a three dimensional computer graphics world by moving his hand is presented. The system can interpret hand motions both in a frame fixed to the world and a frame attached to the user. If the latter is chosen, the user can move the object forward by moving his hand forward even if he has changed his body position. In addition, the user does not have to keep in mind that his hand is in the camera field of view. The active camera system tracks the user to keep him in its field of view. Moreover, the system does not need any camera calibration. The key for the realization of the system with such features is vision algorithms based on the multiple view affine invariance theory. An experimental real-time interface system as well as the vision algorithms are demonstrated. In reality, human operation experiments show the usefulness of the system.