Hand manipulation of virtual objects in wearable augmented reality

This paper proposes an interface for wearable computers using augmented reality (AR) environment which allows the user to handle virtual objects with his/her own hands intuitively. The proposed system is constructed with a wearable computer and a head mounted display (HMD). A pair of cameras is attached to HMD positioned as a pair of eyes. Using the stereo measurement, the finger's postures are captured and displayed together with virtual objects on a video-see-though HMD. The system tracks the fingertips motion of the index finger and the thumb so as the user can pick up, move and rotate the virtual object as if treating a real object with the hands in the real world. While the stereo measurement, range information is added to the region of the hand Visual occlusion is synthesized between the user's hand and the virtual objects including 2D based graphical user interfaces, 3D computer graphics models and so on.