Thermal Videodiff (TVD): A Diffusion Architecture For Thermal Video Synthesis

Machine perception gathers information about its surroundings through sophisticated sensors. Thermal sensors offers scalable perception in conditions of low visibility such as night time, smoke or fog. However, integrating thermal data with deep learning algorithms is a challenge because of scarcity of thermal data due to high cost of thermal sensors. In this paper, we propose for the first time, a deep learning based framework for thermal video synthesis as an affordable alternative to purchasing costly thermal imaging devices. Here, we have introduced diffusion model for estimating thermal videos from videos in the visible spectrum. Results show that our Thermal VideoDiff (TVD) is capable of synthesizing high fidelity video samples and captures temperature variations from thermal data effectively. Our work addresses the challenge posed by the scarcity of thermal data, as well as brings deep learning to the domain of infrared video generation, enabling research and development in infrared domain. The implementation details of the model and fundamentals of thermal imaging are available at https://github.com/Tayeba/TVD.