Transformer End-to-End Optimization of Compressive LiDARs Using Imaging Spectroscopy Side Information

Compressive satellite LiDAR (CS-LiDAR) has been recently introduced as a radically different computational sensing and reconstruction approach for light detection and ranging (LiDAR) sensing of Earth. It is based on NASA’s adaptive wavelength scanning LiDAR (AWSL) system. Rather than measuring 1-D line footprints over a satellite’s swath path as is the norm today, CS-LiDAR adopts sparse coded laser illumination over a 2-D wide field-of-view. The objective is to compressively sense Earth from hundreds of kilometers above Earth to then computationally reconstruct the 3-D imagery with resolution and coverage as if the data were collected from just hundreds of meters in height. The forward imaging model captures the light propagation phenomena affecting the photon pulses transmitted from the sensor to the Earth’s surface and back. This article advances CS-LiDAR on many fronts. First, imaging spectroscopy side information, often jointly available with LiDARs, is integrated into a multimodal imaging system. Second, the inverse imaging problem is cast under a transformer network architecture driven by multimodal attention mechanisms. Finally, by directing the snapshot spectral cameras in front of the LiDAR, the transformer mechanisms autonomously adjust the LiDAR’s beam scanning to focus on specific target locations, thus attaining end-to-end optimal adaptive sampling that can respond to varying observational conditions, surface events, and scientific priorities. Emulations enabled by enormous observational LiDAR data of Earth, available from NASA’s Goddard Lidar, Hyperspectral, Thermal (G-LiHT) imaging observatory, show the advantages attained by methods developed in this work.