Energy-Optimized Trajectory Planning for Solar-Powered Aircraft in a Wind Field Using Reinforcement Learning

High-altitude long-endurance (HALE) solar-powered unmanned aerial vehicles (UAVs) have outstanding advantages in military and civilian fields. In view of the energy management problem of HALE solar aircraft in a wind field, and considering the combined use of solar energy, gravitational potential energy and the wind gradient, this study uses a reinforcement learning (RL) method to train a neural network controller as an integrated aircraft navigation and guidance controller. It is used to optimize the flight trajectory and improve the energy management capability of HALE solar aircraft to enhance long-endurance performance. Simulation experiments of a hover mission show that after 24 hours of flight, the aircraft guided by the RL controller has 3.53% more final total energy, 8.84% more final remaining battery energy and 10.20 kJ more energy from the wind gradient than the L1 state machine. This indicates that the RL controller offers certain advantages in the energy management of HALE solar aircraft and wind gradient energy acquisition compared with the traditional L1 state machine, and this advantage is mainly reflected in the climb and descent stages. The RL controller performs better in acquiring energy from the wind gradient in the descending stage than in the climbing stage. In addition, the robustness is verified.