Blending On-Body and Mid-Air Interaction in Virtual Reality

On-body interfaces, which leverage the human body’s surface as an input or output platform, can provide new opportunities for designing VR interaction. However, it remains unclear how on-body interfaces can best support current VR systems that mainly rely on mid-air interaction. We propose BodyOn, a collection of six design patterns that leverage combined on-body and mid-air interfaces to achieve more effective 3D interaction. Specifically, a user may use thumb-on-finger gestures, finger-on-arm gestures, or on-body displays with mid-air input, including hand movement and orientation, to complete an interaction task. To test our design concepts, we implemented example interaction techniques based on BodyOn that can assist users in various 3D interaction tasks. We further conducted an expert evaluation using the techniques as probes to elicit immediate design issues that emerge from the novel combination of on-body and midair interaction. We provide insights that can inspire and inform the design of future 3D user interfaces.