MAMHESet: A Multi-Angle Monocular Height Estimation Dataset from Luojia3-01 Video Satellite

Reconstructing 3D spatial information from satellite images remains a pivotal challenge in photogrammetry. While traditional methods rely on stereo image pairs, recent advances in monocular height estimation (MHE) have demonstrated the promising feasibility of recovering 3D information without stereo observation conditions. At present, the mainstream deep learning-based MHE models typically learn the mapping between the image-derived deep features and height distribution from extensive RGB-height image pairs. However, the scarcity of satellite MHE datasets severely restricts the application of MHE models in real-world scenarios. In addition, existing MHE datasets predominantly rely on pairs of approximate orthophotography images, which can not be guaranteed in practical satellite imagery acquisitions, undoubtedly leading to the inability of the existing MHE models to learn elevation perception information from varying observation geometries. On the other hand, collecting multi-angle imagery data through cross-orbit or multi-temporal satellite observation introduces significant costs and temporal latency for producing such multi-angle datasets. To this end, this paper constructs the Multi-Angle Monocular Height Estimation Dataset (MAMHESet) to verify the performance of the MHE models when processing multi-angle images. The proposed dataset is constructed using multi-angle images acquired from the Luojia3-01 optical video satellite in gazing mode and the corresponding aerial LiDAR point clouds. We develop a novel dataset generation pipeline, which effectively mitigates dataset construction challenges caused by the inherent heterogeneous point cloud quality differences. To the best of our knowledge, this is the first MHE dataset derived from optical video satellite imagery with multiple observation angles. The extensive training and accuracy evaluations on several mainstream MHE models have demonstrated the practicality of this dataset, which further expands the application potential of the monocular satellite imagery in 3D reconstruction tasks.