Partial-Net: A Method for Data Gaps Reconstruction on Mars Images

Data gaps appear in images of the Martian surface taken by the HiRISE camera on Mars orbiter reconnaissance, affecting downstream missions, e.g., terrain analysis. Recently, many restoration works are based on the standard convolutions, in which the raw features of noise or wrong initialization values of gaps at the encoder will be propagated to the decoder level. This often results in artifacts, such as color differences and blurriness. In addition, existing mask sets commonly used in computer vision cannot simulate and learn the particular irregular shapes of data gaps in Mars images well. To address these issues, we propose a method termed Partial-Net, which is suitable for data gaps filling with respect to the Martian surface images. Partial-Net exclusively utilizes valid pixels and is not contingent on the initial values within the data gap regions due to the implementation of partial convolution. The acquisition of features focusing on valid regions enhances the coherence and naturalness in filling data gaps. The mask self-updating mechanism is applied simultaneously following the partial convolution of each layer, effectively tracking the shape of the mask and reconstructing missing areas during forward propagation. Furthermore, we create a mask set to imitate Mars particular gap areas by extracting masks from missing images of Mars and performing morphological operations on the masked images. We also introduce random free-form masks to improve generalization performance. Experiments on simulated and real damaged Martian surface images show that our method exhibits excellent restoration performance in both overall visual perception quality and local detail texture, which is also helpful for other downstream tasks. For example, for the Mars landmark classification task with usage of the ResNet-34 baseline model, the average accuracy obtained via utilizing our repaired images surpasses via utilizing the original image by 4% (0.82 versus 0.78).