Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration

In collaboration systems, collaborators often use hand gestures for showing information relating to a distant object. However, in viewer's perspective, it is sometimes difficult to know where the conventional gesture cue (Hands-in-Air style: HiA) refers to because the conventional HiA gesture appears away from the distant object. In this paper, we investigate how two factors, distance to the object and view angle difference between collaborators, influence the understanding of HiA gesture by comparing the use of it at 25 positions with 5 distances and 5 view angles. In a user study, we found that the distance to the target object and view angle difference negatively influenced HiA gesture communication. The influence of the distance and view angle was more serious between smaller angles (0~30 degree) than larger angles (30~60 degree), and between shorter distances (1m~1.5m) than between longer distance (1.5m~2m). As a solution, we propose the Hands-on-Target (HoT) style which positions the hand gesture cue on the surface of the target object. The HoT style gesture cue dramatically reduces the negative effect of the distance and view angle difference. Participants completed the task 29.3 percent faster, selected the correct object 2.75 times more, and felt 63.1 percent less mental effort. For further investigation, we discuss on the extensibility of the HoT interface that it can be used not only for object selection but also for diverse type of gesture communication including gesture for object manipulation because the HoT interface can support all possible real world hand gestures.