Deep Synthesis of Cloud Lighting

Current appearance models for the sky are able to represent clear-sky illumination to a high degree of accuracy. However, these models all lack a common feature of real skies: clouds. These are an essential component for many applications which rely on realistic skies, such as image editing and synthesis. While clouds can be added to existing sky models through rendering, this is hard to achieve due to the difficulties of representing clouds and the complexities of volumetric light transport. In this work, an alternative approach to this problem is proposed whereby clouds are synthesized using a learned data-driven representation. This leverages a captured collection of high dynamic range cloudy sky imagery, and combines this dataset with clear-sky models to produce plausible cloud appearance from a coarse representation of cloud positions. This representation is artist controllable, allowing for novel cloudscapes to be rapidly synthesized, and used for lighting virtual environments.