Video Generation with Single Flame Image and Wind Direction Information

In recent years, the shortage of human resources in the animation and video production industry has become a sig-nificant issue, leading to a demand for efficient video generation methods. Deep learning-based video generation techniques are seen as a potential solution, with research focusing on generating natural phenomena videos from single images. Progress has been made in developing video generation networks that incorporate external forces, such as wind, into the models. However, there is still a need for further advancements in creating networks that can more accurately and effectively respond to the creator's intent and handle complex natural phenomena. In this paper, we constructed a video generation network that uses a single image of a flame and wind direction information as input to generate natural flame videos considering the external force of wind. This network was trained through a two-stage process based on VQ-VAE (Vector Quantized-Variational AutoEncoder) and Transformer models. The network demonstrated the ability to capture the relationship between flame motion and wind. The videos generated by this network accurately depicted changes in flame behavior due to wind direction, showcasing the high accuracy of the video generation. Furthermore, the Frechet Video Distance (FVD) index achieved by our network was comparable to those reported in similar studies, indicating competitive performance.