Effects of visual latency on visual-haptic experience of stiffness

In multimodal virtual-reality, augmented-reality, and tele-operation systems, a temporal asynchrony often exists between visual and haptic feedback due to differences in processing and rendering the two types of signals. We have conducted two psychophysical experiments to examine how such asynchrony influences our perceptual experience with an object's stiffness. Participants explored a virtual elastic material using a haptic interface, and saw the deformation of the material in a simulated ultrasound that was displayed with a constant or variable latency relative to the haptic feedback. Their perception of stiffness and ability to differentiate stiffness were measured. The results showed that the perceived stiffness increased with the visual latency while the differentiation threshold was little influenced. When the visual latency was variable, the effects were reduced and participants relied more on the haptic sensations. Such effects will be further evaluated in clinical settings using a neural surgical simulator.