Scene Transformer: Automatic Transformation from Real Scene to Virtual Scene

Given a real scene and a virtual scene, the indoor scene transformation problem is defined as transforming the layout of the input virtual scene. The transformed layout preserves as much as possible the relationship between the furniture in the input virtual scene, and the input real scene provides the user with as much passive haptic as possible when exploring the virtual scene. We propose a real-scene-constrained deep scene transformer to solve this problem. First, we introduce the deep scene matching network to predict the matching relationship between real furniture and virtual furniture. Then we introduce a layout refinement algorithm based on the refinement parameter network to arrange the matched virtual furniture into the new virtual scene. At last, we introduce a deep scene generating network to arrange the unmatched virtual furniture into the new virtual scene.