Reinforcement Learning-Based Autonomous Aircraft Navigation with Potential-Based Reward Shaping

Autonomous navigation is essential to promoting the autonomy of the aircraft. While adjusting the velocity direction and altitude angles to navigate to the goal, aircraft must maintain speed to prevent losing height. In this research, a deep reinforcement learning method with a potential-based dense reward function is proposed to teach the agent to navigate independently. The method integrates the potential-based dense reward function to help the PPO algorithm with the navigation task. Experiments are conducted among three different reward functions: sparse reward, dense reward, and potential-based dense reward. The results show that the sparse reward can rarely guide the agent to navigate, whereas the rest can complete the assignment. Furthermore, the agent with the potential-based dense reward has a 15% greater task success rate than the agent with the dense reward. Finally, a validation experiment is carried out to see if the well-trained agent, which excels at navigating to a fixed target point, can track another aircraft that maintains a constant path and speed. It is inferred that the proposed method is capable of not only navigating to a fixed target location but also tracking another aircraft with consistent course and speed.