The Effects of Audio and Haptic Feedback on Collaborative Scanning and Placing

This paper presents a study aimed at exploring the effects of different modality combinations on collaborative task performance and employed joint task-solving strategies in a shared interface. The modality combinations visual/haptic, visual/audio and visual/haptic/audio were compared in an experiment in which users solved a task together, working in pairs in adjacent rooms. The application used contained a flat surface in a 3D interface on which piles of cubes were randomly placed in a grid. The task involved scanning for empty cells and placing continuously falling cubes until all empty cells were filled. The cubes and the flat surface were designed in such a way that they could be felt and heard and thus could be recognized by different kinds of haptic and audio feedback cues. This made it possible to scan the environment and read both absolute and relative positions in the grid. A quantitative analysis of task performance and a qualitative analysis of video recordings and interview data were performed. Results showed that task completion times were significantly faster in the visual/haptic/audio condition compared with the other conditions and that there were also significantly fewer errors, result checks of one’s own actions and double checks of the partner’s actions in the visual/haptic/audio condition than in the other conditions. Qualitative results show that participants work simultaneously to a larger extent in the visual/haptic/audio condition and that less communication occurred in the visual/haptic/audio condition compared with the other conditions. We argue that more modalities improved the awareness of the environment resulting in the participants feeling more confident with their interaction in the environment in the visual/haptic/audio condition. This resulted in improved task performance. The visual/audio feedback was better suited for solving the task than the visual/haptic feedback even though haptic feedback gave a significant added value in the visual/haptic/audio condition.