A Lightweight Hybrid Representation for Virtual Complex Scenes

While virtual scenes with complex geometries and highresolution physically based rendering(PBR) material textures are capable of representing details, they often occupy a large amount of memory and require significant manual editing by artists. In this paper, we introduce a new framework for representing virtual scenes more efficiently. Our framework utilizes a lightweight hybrid representation that combines traditional polygon-based meshes and material textures with learned neural textures. By using simplified geometries and lower-resolution PBR material textures, our representation achieves a compact size while maintaining the freedom for user-friendly scene editing of materials and illuminations. Additionally, we have designed a hybrid rendering pipeline to render the lightweight representation with ray-traced visual quality at real-time frame rates. Our approach reduces memory requirements, eliminates the need for manual editing, and is easy to train. We demonstrate the effectiveness of our framework on a variety of virtual complex scenes.