DEM-Assisted Neural Network for SAR-to-Optical Image Translation

SAR-to-optical remote sensing translator neural networks are mostly trained on flat areas, avoiding SAR geometrical distortion issues in steeply sloped areas. Their degraded performance under such topology severely limits the ability to detect disasters such as landslides in cloud covered areas. In this paper, we first propose a new SAR-DEM-optical dataset in mountainous regions to improve the performance of SAR-to-optical image translators under these extreme conditions. Then we upgrade SARDINet (SAR Distorted Image translator Network) model previously developed for urban areas, to take a Digital Elevation Model (DEM) together with the SAR image as input and perform translation in a natural mountain environment. Several fusion strategies are explored to efficiently merge SAR and DEM images: late fusion, early fusion and an intermediate fusion based on balanced separable convolutions. These approaches show improvements in distorted regions compared to the original SARDINet and two standard adversarial networks - Pix2pix and CycleGAN.