LMNav: Landmark-Centric Multimodal Reasoning for Vision-And-Language Navigation in Continuous Environments

This paper presents LMNav, a landmark-centric navigation framework for Vision-And-Language Navigation in Continuous Environments (VLN-CE). Inspired by the way humans navigate using landmarks and their spatial relationships, LMNav explicitly extracts semantic landmarks and their temporal order from natural language instructions via a language model, constructing a structured directed graph. This graph guides both the agent’s reasoning and perception processes. To enhance visual understanding, LMNav employs view expansion and a semantic-depth fusion module that integrates semantic segmentation and depth cues using an SE attention mechanism. Landmark features are further modulated by a dynamic masking mechanism based on distance, and then fused with RGB embeddings via a cross-attention module. These rich multimodal representations are injected into the policy network to improve decision-making. We evaluate LMNav on the R2R-CE benchmark using standard VLN metrics. The proposed model achieves a success rate of 51.6% and an SPL of 43.0, while also reducing trajectory length compared to baselines. The results demonstrate that LMNav enables more accurate and efficient navigation by leveraging structured landmark reasoning and cross-modal integration.