Merging Camera and Object Haptic Motion Effects for Improved 4D Experiences

(Haptic) motion effects refer to the vestibular stimuli generated by a motion platform and delivered to the whole body of a user sitting on the platform. Motion effects are an essential tool for creating vivid sensory experiences in various extended reality (XR) applications, ranging from training simulators to recent 4D rides, films, and games for entertainment. For the latter purpose, motion effects emphasize audiovisual events occurring in the scene, such as camera motion, the movement of an object of interest, and special sounds. Recent research developed several algorithms to produce motion effects from the audiovisual stream automatically. However, these algorithms are designed for a single class of motion effects, and extension to multiple motion effect classes remains unexplored. In this paper, we propose an algorithmic framework that merges camera and object motion effects into one motion effect while preserving the perceptual consequences of the two effects. We validate the framework’s perceptual performance through a user study. To our knowledge, this work is one of the first successful reports of merging different kinds of motion effects for improved XR experiences.