DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal From Optical Satellite Images

Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis. Consequently, effectively removing clouds from optical satellite images has emerged as a prominent research direction. Recent advances in deep learning-based cloud removal methods have been significant, but the quality of image generation still needs to be improved. Diffusion models have demonstrated remarkable success in diverse image generation tasks, showcasing their potential in addressing this challenge. This article presents a novel framework called DiffCR, which leverages conditional guided diffusion with deep convolutional networks for high-performance cloud removal for optical satellite imagery. Specifically, we introduce a decoupled encoder for conditional image feature extraction, providing a robust color representation to ensure the close similarity of appearance information between the conditional input and the synthesized output. Moreover, we propose a novel and efficient time and condition fusion block (TCFBlock) within the cloud removal model to accurately simulate the correspondence between the appearance in the conditional image and the target image at a low computational cost. Extensive experimental evaluations on three commonly used benchmark datasets demonstrate that DiffCR consistently achieves the state-of-the-art (SOTA) performance on all metrics, with parameter and computational complexities amounting to only 5.1% and 5.4%, respectively, of those previous best methods. The source code, pretrained models, and all the experimental results will be publicly available at
https://github.com/XavierJiezou/DiffCR
upon the paper’s acceptance of this work.