Distributed Haptic Interactions with Physically Based 3D Deformable Models over Lossy Networks

Researchers have faced great challenges when simulating complicated 3D volumetric deformable models in haptics-enabled collaborative/cooperative virtual environments (HCVEs) due to the expensive simulation cost, heavy communication load, and unstable network conditions. When general network services are applied to HCVEs, network problems such as packet loss, delay, and jitter can cause severe visual distortion, haptic instability, and system inconsistency. In this paper, we propose a novel approach to support haptic interactions with physically based 3D deformable models in a distributed virtual environment. Our objective is to achieve real-time sharing of deformable and force simulations over general networks. Combining linear modal analysis and corotational methods, we can effectively simulate physical behaviors of 3D objects, even for large rotational deformations. We analyze different factors that influence HCVEs' performance and focus on exploring solutions for streaming over lossy networks. In our system, 3D deformation can be described by a fairly small amount of data (several KB) using accelerations in the spectral domain, so that we can achieve low communication load and effective streaming. We develop a loss compensation and prediction algorithm to correct the errors/distortions caused by network problem, and a force prediction method to simulate force at users' side to ensure the haptic stability, and the visual and haptic consistency. Our system works well under both the client-server and the peer-to-peer distribution structures, and can be easily extended to other topologies. In addition to theoretical analysis, we have tested the proposed system and algorithms under various network conditions. The experimental results are remarkably good, confirming the effectiveness, robustness, and validity of our approach.