Exploring Visual Techniques for Boundary Awareness During Interaction in Augmented Reality Head-Mounted Displays

Mid-air hand interaction has long been proposed as a ‘natural’ input method for Augmented Reality (AR) systems. Current AR Head-Mounted Displays (HMDs) have a limited area for hand-based interactions. Because of this, users may easily move their hand(s) outside this tracked area during interaction, especially in dynamic tasks (e.g., when translating an object). Compared to common midair interaction issues, such as gesture recognition, arm/hand fatigue, and unnatural ways of interacting with virtual objects (e.g., selecting a distant object), boundary awareness issues in AR devices have received little attention. In this research, we explore visual techniques for boundary awareness in AR HMDs, focusing on object translation tasks. Through a systematic formative study, we first identify the challenges that users might face when interacting with AR HMDs without any boundary awareness information (i.e., how current systems work). Based on the findings, we then propose four methods (i.e., static surfaces, dynamic surface(s), static coordinated lines, and dynamic coordinate line(s)) and evaluate them against the benchmark (i.e., baseline condition without boundary awareness) to make users aware of the tracked interaction area. Our results show that visual methods for boundary awareness can help with dynamic mid-air hand interactions in AR HMDs, but their effectiveness and application are user-dependent.