Cloud Removal With SAR-Optical Data Fusion Using a Unified Spatial–Spectral Residual Network

Cloud contamination greatly limits the potential utilization of optical images for geoscience applications. An effective alternative is to extract data from synthetic aperture radar (SAR) images to remove clouds due to the strong penetration ability of microwaves. In this article, we propose a novel unified spatial–spectral residual network that utilizes SAR images as auxiliary data to remove clouds from optical images. The method can better establish the relationship between SAR and optical images and be divided into two modules: feature extraction and fusion module and reconstruction module. In the feature extraction and fusion module, a gated convolutional layer is introduced to discriminate cloud pixels from clean pixels, which makes up for the lack of distinguishing ability of vanilla convolutional layers and avoids the error of cloud areas in feature extraction. In the reconstruction module, spatial and channel attention mechanisms are introduced to obtain global spatial and spectral information. The network is tested on three datasets with different spatial resolutions and compositions of land covers to verify the effectiveness and applicability of the method. The results show that the method outperforms other mainstream algorithms that simultaneously use SAR images as auxiliary data with a gain of about 2.3 dB in terms of peak signal-to-noise ratio PSNR on the SEN12MS-CR dataset.

I. Introduction

Numerous remote sensing data have been acquired and have been used for geoscience applications to facilitate the monitoring of the land surface environment [1]. However, the optical imagery acquired by satellite sensors is inevitably contaminated by thick clouds and their shadows, which enormously increases the difficulty of further applications. The global cloud amount provided by the International Satellite Cloud Climate Program (ISCCP) shows that 63% of the Earth is covered by clouds [2]. Cloud cover is a common problem in optical satellite imagery, which will lead to large quantities of missing information and cause great difficulties in subsequent image interpretation and application. Thus, reconstructing cloud-free images through cloud removal methods is much sought after.