Hidden-Surface Processing for Interaction of Hand and Virtual Objects Using Leap Motion Controller

Recently augmented reality (AR) technology has attracted a lot of attention and is studied. Furthermore, it is applied to the interaction technology between human and virtual objects. In AR interaction technology, it is necessary for the user to be able to manipulate virtual objects intuitively, so it is required to manipulate the objects by their hand. However, in AR technology, since the 3-dimensional model is superimposed on the image of the real space afterwards, it is always displayed on the front side than the hand (occlusion problem). It is an unnatural scene in some cases. In this study, we utilized Leap Motion Controller to acquire the depth data of the user's hand and fingers. We performed hidden-surface processing considering the depth and the color of the hand and fingers. Thus we enabled appropriate hidden-surface processing in the area of the user's hand and fingers.