Map-SemNav: Advancing Zero-Shot Continuous Vision-and-Language Navigation Through Visual Semantics and Map Integration

This paper explores zero-shot Vision-and-Language Navigation (VLN), enabling agents to generalize navigation to unseen data classes. Most current approaches rely on large models, but these are not specifically tailored for VLN, lacking direct learning from navigation environments and slowing down agents due to their overwhelming size. To tackle this, we propose Map-Semantic Zero-shot Navigation (Map-SemNav), which does not rely on large models for navigation planning. Map-SemNav utilizes three key cues: direction, object, and scene, to acquire relational knowledge instead of memorizing specific classes, which enables generalization to unseen data. Direction is guided by a top-down semantic map, while object and scene information is decoupled from environment knowledge. Extensive experiments demonstrate that Map-SemNav outperforms state-of-the-art large model-based methods in zero-shot VLN tasks within continuous environments, while also offering higher efficiency due to its simplified architecture.