Visuo-Somatosensory Interaction for Space Telescience of Fluid Experiments

In this paper, we propose a novel visuo-somatosensory interaction strategy for space telescience experiments to provide a realistic 'virtual presence' at the space experiment workplace in a sequential multi-step process. We provide background on the current available tools for telescience experiment and discuss the requirements when performing the liquid bridge experiment. We model the fluid as physical-based data-driven dynamic particles and develop to display in 3D stereoscopic scenario using the Oculus Rift. We design the task-command hierarchical interaction solution for less mental workload and the recognition algorithm of a series of understandable hand(s) gestures for intuitive control using Leap Motion. We introduce tactile-based augmentation interaction to provide a helpful guide. Finally, the results show the proposed method can provide a more efficient, natural and intuitive user experience compared with traditional interaction. For space experiment of liquid bridge, two-hand gestures show the higher accuracy and less mental workload than the single-hand gesture and the variant gesture. Also, a pilot study of gesture-based control with tactile feedback is validated to be a potentially powerful solution to enhance the user's interaction experience.