Generative Shadow Synthesis and Removal for Remote Sensing Images Through Embedding Illumination Models

Shadows significantly reduce the available information in remote sensing images, obstructing downstream tasks such as object detection, scene classification, and localization. Shadow removal from remote sensing images is, however, still an open issue for the following reasons. First, deep neural networks are difficult to train since the corresponding ground truth of shadows is almost always unavailable in practice. Second, the existing shadow removal methods still suffer from blurry details and boundary artifacts. In this article, we describe how a generative shadow synthesis and removal framework that couples data-driven methods with illumination models was developed to address the above challenges effectively. Various shadows were synthesized in shadow-free regions of remote sensing images by GSS-Net, which is a generative shadow synthesis network that considers the physical process of shadow illumination attenuation (SIA). In this way, a large-scale, diverse, and realistic shadow dataset (RS-SynShadow) was built. A generative shadow removal network—GSR-Net—embedding a histogram-enhanced illumination (HEI) model was then developed for high-fidelity shadow removal without artifacts. Extensive experiments conducted on synthetic and real data demonstrate that the proposed shadow synthesis and removal framework significantly outperforms the state-of-the-art methods, both visually and quantitatively. The dataset and code will be made available at https://github.com/fzzfRS/RS-GSSR