HAFE: Hierarchical Attention-based Frontier Exploration for Multi-Robot Mapping

Autonomous exploration is a critical application of multi-vehicle systems, where a team of networked robots collaboratively explores an unknown environment. This technique is crucial for applications like search and rescue, fault detection, and mapping. Traditional frontier-based methods struggle with cooperative exploration, while multi-agent deep reinforcement learning (MARL) approaches suffer from inefficient task assignments due to shortsighted decision-making and lack of long-term planning. To address these limitations, we propose HAFE (Hierarchical Attention-based Frontier Exploration), a hierarchical reinforcement learning (HRL) framework that integrates temporal decision modeling into multi-robot exploration. The high-level policy, built on an LSTM-Attention network, captures long-term dependencies in exploration history and dynamically allocates goal frontiers for each robot, ensuring structured and non-redundant coverage. The low-level MADDPG policy then executes navigation and obstacle avoidance. Additionally, we introduce frontier utility embedding into the state representation, enabling robots to prioritize unexplored regions more effectively. Experimental results in Gazebo and ROS demonstrate that HAFE significantly outperforms MADDPG, achieving faster coverage, higher rewards, and fewer collisions by leveraging historical information for more informed goal selection.