RLoPlanner: Combining Learning and Motion Planner for UAV Safe Navigation in Cluttered Unknown Environments

In this paper, we propose a hierarchical navigation framework named RLoPlanner that combines deep reinforcement learning algorithms and local motion planners, allowing unmanned aerial vehicles (UAVs) to perform navigation tasks safely and energy-efficiently in complex and unknown environments. This technique is critical to improve the performance of UAVs in environments without prior maps. Specifically, the framework high-level generates a stochastic policy through a deep reinforcement learning algorithm based on maximum entropy, which generates a local target based on raw sensor information. Then the low-level motion planner tracks the local goal to generate a smooth trajectory to the final target. Compared with existing end-to-end navigation methods, the proposed navigation framework generates trajectories that are smoother, more energy-efficient and more dynamically feasible. The framework also overcomes the drawback that the mapping and planning methods tend to fall into local minima. Our experiments in a simulated environment with random obstacles demonstrate that RLoPlanner outperforms state-of-the-art methods in terms of navigation success rate and kinematically compliant trajectories.