Real-time Bi-directional Real-Virtual Interaction Framework Using Automatic Simulation Model Generation

Most mixed reality researches require lots of event-specific interaction, such as assigning appropriate animations for each specific situation, without considering the physical properties of objects during the modeling process. We propose a new framework for bi-directional interaction between real and virtual objects that in-corporates physical feedback on both real and virtual objects. Our framework automatically generates simulation-enabled virtualized real objects using only their 3D meshes and material types related to their physical properties and visualizes in real-time various bidirectional interactions among virtual and real objects having various types of physical properties such as rigid, articulated rigid, and deformable body.