Geometrically-plausible and Semantically-consistent Generation of Indoor Panoramas

We present PanoGPS, a new approach for generating indoor panoramas, capable of following a room layout sketch and description text to generate panoramas with geometrically-plausible room layouts and semantically-consistent contents. Overall, our idea is to inform the generative model collectively with the two inputs, to inspire it to implicitly generate and refine a unified latent code for panorama generation. Specifically, we first propose using the semantic layout map to encode the room geometry to condition the generative model with plausible geometric information. Second, we establish a pipeline to integrate features from the two inputs and generate a single unified latent code that describes both the room layout and room contents of the target panorama. Third, we progressively refine the latent code to produce a coarse panorama to enforce seamless left-right border connections, while iteratively upsizing the coarse panorama with details. Our method outperforms existing approaches by generating high-quality panoramas with geometrically plausible structures and semantically meaningful content. Code is available at: https://github.com/wumengyangok/PanoGPS