Abstract:
The existence of thick clouds covers the comprehensive Earth observation of optical remote sensing images (RSIs). Cloud removal is an effective and economical preprocessi...Show MoreMetadata
Abstract:
The existence of thick clouds covers the comprehensive Earth observation of optical remote sensing images (RSIs). Cloud removal is an effective and economical preprocessing step to improve the subsequent applications of RSIs. Deep learning (DL)-based methods have attracted much attention and achieved state-of-the-art results. However, most of these methods suffer from the following issues: 1) ignore the physical characteristics of RSIs; 2) require paired images with/without cloud or extra auxiliary images; and 3) demand the cloud mask. These issues might have limited the flexibility of existing networks. In this article, we propose a novel low-rank regularized self-supervised network (LRRSSN) that couples model-driven and data-driven methods to remove the thick cloud from multitemporal RSIs (MRSIs). First, motivated by the equal importance of image and cloud components as well as their intrinsic characteristics, we decompose the observed image into low-rank image and structural sparse cloud components. In this way, we obtain a model-driven thick cloud removal method where the spectral–temporal low-rank correlation of the image component and the spectral structural sparsity of the cloud component are effectively exploited. Second, to capture the complex nonlinear features of different scenarios, the data-driven self-supervised network that does not require external training datasets is designed to explore the deep prior of the image component. Third, the coupled model-driven and data-driven LRRSSN is optimized by an efficient half quadratic splitting (HQS) algorithm. Finally, without knowing the exact cloud mask, we estimate the cloud mask to preserve information in cloud-free areas as much as possible. Experiments conducted in synthetic and real-world scenarios demonstrate the effectiveness of the proposed approach.
Published in: IEEE Transactions on Geoscience and Remote Sensing ( Volume: 62)
Funding Agency:

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Yong Chen received the B.S. degree from the School of Science, East China University of Technology, Nanchang, China, in 2015, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2020.
From 2018 to 2019, he was a Research Intern with the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is curr...Show More
Yong Chen received the B.S. degree from the School of Science, East China University of Technology, Nanchang, China, in 2015, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2020.
From 2018 to 2019, he was a Research Intern with the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is curr...View more

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Maolin Chen received the B.S. degree from Jiangxi Normal University, Nanchang, China, in 2022, where he is currently pursuing the M.S. degree with the School of Computer and Information Engineering.
His research interests include cloud removal and deep learning.
Maolin Chen received the B.S. degree from Jiangxi Normal University, Nanchang, China, in 2022, where he is currently pursuing the M.S. degree with the School of Computer and Information Engineering.
His research interests include cloud removal and deep learning.View more

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China
Wei He (Senior Member, IEEE) received the B.S. degree from the School of Mathematics and Statistics, Wuhan University, Wuhan, China, in 2012, and the Ph.D. degree from the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, in 2017.
From 2018 to 2020, he was a Researcher at the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japa...Show More
Wei He (Senior Member, IEEE) received the B.S. degree from the School of Mathematics and Statistics, Wuhan University, Wuhan, China, in 2012, and the Ph.D. degree from the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, in 2017.
From 2018 to 2020, he was a Researcher at the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japa...View more

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Jinshan Zeng (Member, IEEE) received the Ph.D. degree in mathematics from Xi’an Jiaotong University, Xi’an, China, in 2015.
He is currently a Professor with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China, where he has been serving as the Associate Dean since July 2022. He has authored over 60 paperss in high-impact journals and conferences such as Journal of Machine Learning ...Show More
Jinshan Zeng (Member, IEEE) received the Ph.D. degree in mathematics from Xi’an Jiaotong University, Xi’an, China, in 2015.
He is currently a Professor with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China, where he has been serving as the Associate Dean since July 2022. He has authored over 60 paperss in high-impact journals and conferences such as Journal of Machine Learning ...View more

School of Geography and Environment, Jiangxi Normal University, Nanchang, China
Min Huang received the B.Sc. degree in remote sensing science and technology and the Ph.D. degree in cartography and geographical information system from Wuhan University, Wuhan, China, in 2015 and 2021, respectively.
He is currently a Lecturer with the School of Geography and Environment, Jiangxi Normal University, Nanchang, China. His research interests include urban remote sensing, smart cities, land use/land cover clas...Show More
Min Huang received the B.Sc. degree in remote sensing science and technology and the Ph.D. degree in cartography and geographical information system from Wuhan University, Wuhan, China, in 2015 and 2021, respectively.
He is currently a Lecturer with the School of Geography and Environment, Jiangxi Normal University, Nanchang, China. His research interests include urban remote sensing, smart cities, land use/land cover clas...View more

School of Information Science and Technology, Southwest Jiaotong University, Chengdu, China
Yu-Bang Zheng (Member, IEEE) received the B.S. degree from the Institute of Statistics and Applied Mathematics, Anhui University of Finance and Economics, Bengbu, China, in 2017, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China, in 2022.
From 2021 to 2022, he was a Student Trainee with the Tensor Learning Team, RIKEN Center for Advanced ...Show More
Yu-Bang Zheng (Member, IEEE) received the B.S. degree from the Institute of Statistics and Applied Mathematics, Anhui University of Finance and Economics, Bengbu, China, in 2017, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China, in 2022.
From 2021 to 2022, he was a Student Trainee with the Tensor Learning Team, RIKEN Center for Advanced ...View more

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Yong Chen received the B.S. degree from the School of Science, East China University of Technology, Nanchang, China, in 2015, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2020.
From 2018 to 2019, he was a Research Intern with the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is currently working with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang. His research interests include hyperspectral image processing, low-rank matrix/tensor representation, and model-driven deep learning.
Yong Chen received the B.S. degree from the School of Science, East China University of Technology, Nanchang, China, in 2015, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2020.
From 2018 to 2019, he was a Research Intern with the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is currently working with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang. His research interests include hyperspectral image processing, low-rank matrix/tensor representation, and model-driven deep learning.View more

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Maolin Chen received the B.S. degree from Jiangxi Normal University, Nanchang, China, in 2022, where he is currently pursuing the M.S. degree with the School of Computer and Information Engineering.
His research interests include cloud removal and deep learning.
Maolin Chen received the B.S. degree from Jiangxi Normal University, Nanchang, China, in 2022, where he is currently pursuing the M.S. degree with the School of Computer and Information Engineering.
His research interests include cloud removal and deep learning.View more

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China
Wei He (Senior Member, IEEE) received the B.S. degree from the School of Mathematics and Statistics, Wuhan University, Wuhan, China, in 2012, and the Ph.D. degree from the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, in 2017.
From 2018 to 2020, he was a Researcher at the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan, where he was a Research Scientist from 2020 to 2021. He is currently a Full Professor with LIESMARS, Wuhan University. His research interests include image quality improvement, remote sensing image processing and low-rank representation, and deep learning.
Wei He (Senior Member, IEEE) received the B.S. degree from the School of Mathematics and Statistics, Wuhan University, Wuhan, China, in 2012, and the Ph.D. degree from the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, in 2017.
From 2018 to 2020, he was a Researcher at the Geoinformatics Unit, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan, where he was a Research Scientist from 2020 to 2021. He is currently a Full Professor with LIESMARS, Wuhan University. His research interests include image quality improvement, remote sensing image processing and low-rank representation, and deep learning.View more

School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China
Jinshan Zeng (Member, IEEE) received the Ph.D. degree in mathematics from Xi’an Jiaotong University, Xi’an, China, in 2015.
He is currently a Professor with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China, where he has been serving as the Associate Dean since July 2022. He has authored over 60 paperss in high-impact journals and conferences such as Journal of Machine Learning Research (JMLR), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), IEEE Transactions on Signal Processing (TSP), IEEE Transactions on Image Processing (TIP), IEEE Transactions on Neural Networks and Learning Systems (TNNLS), IEEE Transactions on Geoscience and Remote Sensing (TGRS), ICML, and AAAI. He has had two papers coauthored with collaborators that received the International Consortium of Chinese Mathematicians (ICCM) Best Paper Award in 2018 and 2020. His research interests include nonconvex optimization, machine learning, remote sensing, and computer vision.
Jinshan Zeng (Member, IEEE) received the Ph.D. degree in mathematics from Xi’an Jiaotong University, Xi’an, China, in 2015.
He is currently a Professor with the School of Computer and Information Engineering, Jiangxi Normal University, Nanchang, China, where he has been serving as the Associate Dean since July 2022. He has authored over 60 paperss in high-impact journals and conferences such as Journal of Machine Learning Research (JMLR), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), IEEE Transactions on Signal Processing (TSP), IEEE Transactions on Image Processing (TIP), IEEE Transactions on Neural Networks and Learning Systems (TNNLS), IEEE Transactions on Geoscience and Remote Sensing (TGRS), ICML, and AAAI. He has had two papers coauthored with collaborators that received the International Consortium of Chinese Mathematicians (ICCM) Best Paper Award in 2018 and 2020. His research interests include nonconvex optimization, machine learning, remote sensing, and computer vision.View more

School of Geography and Environment, Jiangxi Normal University, Nanchang, China
Min Huang received the B.Sc. degree in remote sensing science and technology and the Ph.D. degree in cartography and geographical information system from Wuhan University, Wuhan, China, in 2015 and 2021, respectively.
He is currently a Lecturer with the School of Geography and Environment, Jiangxi Normal University, Nanchang, China. His research interests include urban remote sensing, smart cities, land use/land cover classification, surface parameter inversion, multisourced data fusion, and geospatial bbreak big data analysis.
Min Huang received the B.Sc. degree in remote sensing science and technology and the Ph.D. degree in cartography and geographical information system from Wuhan University, Wuhan, China, in 2015 and 2021, respectively.
He is currently a Lecturer with the School of Geography and Environment, Jiangxi Normal University, Nanchang, China. His research interests include urban remote sensing, smart cities, land use/land cover classification, surface parameter inversion, multisourced data fusion, and geospatial bbreak big data analysis.View more

School of Information Science and Technology, Southwest Jiaotong University, Chengdu, China
Yu-Bang Zheng (Member, IEEE) received the B.S. degree from the Institute of Statistics and Applied Mathematics, Anhui University of Finance and Economics, Bengbu, China, in 2017, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China, in 2022.
From 2021 to 2022, he was a Student Trainee with the Tensor Learning Team, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is currently working with the School of Information Science and Technology, Southwest Jiaotong University, Chengdu. His research interests include tensor modeling and computing, tensor learning, and high-dimensional data processing.
Yu-Bang Zheng (Member, IEEE) received the B.S. degree from the Institute of Statistics and Applied Mathematics, Anhui University of Finance and Economics, Bengbu, China, in 2017, and the Ph.D. degree from the School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China, in 2022.
From 2021 to 2022, he was a Student Trainee with the Tensor Learning Team, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. He is currently working with the School of Information Science and Technology, Southwest Jiaotong University, Chengdu. His research interests include tensor modeling and computing, tensor learning, and high-dimensional data processing.View more