STSR-Nav: Spatial-Temporal Scene Representation for Navigation Policy Learning Based on UAV-UGV Collaborative Perception

In UAV-UGV collaboration, UGVs enhance their perception capabilities by utilizing shared observation data from UAVs to make effective decisions. However, learning navigation policies directly from multi-perspective observation data is challenging due to spatial inconsistencies and redundancy. Existing approaches either rely on low-level scene representations, which may reduce efficiency, or use mid-level semantic representations that lack essential geometric and temporal details, resulting in sub-optimal decisions. This paper presents STSR-Nav, a novel framework for navigation policy learning based on UAV-UGV collaborative perception. The framework first encodes semantic information and geometric relationships from UAV images using a Bird'View (BEV) scene graph. A learnable graph neural network then integrates the BEV scene graph with local perception to infer spatial features. Temporal learning is applied to these features via a memory network, generating spatial-temporal features that are embedded into the UGV's local perception space using a multi-head attention mechanism. Finally, the fused spatial-temporal features provide a high-level scene representation for navigation policy learning. Unity simulations demonstrate the framework's improved efficiency and generalization capabilities.