Effects of sensory feedback while interacting with graphical menus in virtual environments

The present study investigates the effect of three types of sensory feedback (visual, auditory and passive haptic) in a context of two-handed interaction with graphical menus in virtual environments. Subjects controlled the position and orientation of a graphical menu using their non-dominant hand and interacted with menu items using their dominant index fingertip. An ISO 9241–9-based multi-tapping task and a sliding task were respectively used to evaluate subjects' performance in different feedback conditions. Adding passive haptic to visual feedback increased movement time and error rate, decreased throughput in the multi-tapping task, but outperformed visual only and visual-auditory feedback in the sliding task (in terms of movement time and number of times the contact between the finger and the pointer was lost). The results also showed that visual-auditory feedback, even if judged as useful by some subjects, decreased users' performance in the sliding task, as compared to visual-only feedback.