DiffSmoke: Two-Stage Sketch-Based Smoke Illustration Design Using Diffusion Models

Due to the rapid explosion of generative artificial intelligence (AI) techniques, generative models are revolutionizing content design applications. However, it is still a challenging task to generate smoke design illustrations due to the complex dynamics and intricate constraints. Previous studies for deep-learning-based fluid design commonly adopted conditional generative adversarial networks (cGAN) for target velocity field generation under the constraint of input sketches, which may behave in unstable training and miss the hidden geometric structure and flow properties inherent in the complex smoke dynamics. To solve these issues, we propose DiffSmoke, a smoke illustration generation method using the two-stage latent diffusion model. In the first stage, sketch inputs serve as the generator condition for generating the Lagrangian Coherent Structure (LCS) region, which correlates with the hyper finite time Lyapunov exponent (hyper-FTLE) field to depict the flow properties. In the second stage, we use LCS data in the generator for the velocity field generation. The evaluation results show that DiffSmoke can generate velocity fields matching the shapes of given sketches. In addition, DiffSmoke enables users to conduct smoke design from hand-drawn sketches and achieve more robust and stable generation results in contrast to cGAN and one-stage strategy.