Artificial Intelligence in UAV Flight Controls: Deep Reinforcement Learning Based Altitude-Hold Strategies for Fixed-Wing UAVs

This paper implements a deep reinforcement learning (DRL) based flight control system for a fixed-wing uncrewed aerial vehicle (UAV). Unlike conventional flight control methods, DRL does not require an exact mathematical system model for the design and can handle non-linear coupled dynamics of highly agile aerospace vehicles like small-size UAVs. The deep deterministic policy gradient (DDPG) method was chosen to suit environments with continuous action spaces. The key contribution of this research is the implementation of three distinct approaches that successfully replace traditional classical control systems with Reinforcement Learning (RL)-based controllers, each offering unique advantages and exploring different trade-offs between interpretability, complexity, and performance crucial for safety-critical aerospace applications. The classical Proportional-Integral-Derivative (PID) flight control architecture, consisting of an altitude controller followed by a pitch (theta) controller, was developed as a baseline. Subsequently, three approaches were investigated; First, the altitude hold controller was replaced by a Reinforcement Learning (RL) agent, while the PID control was maintained for pitch control. In the second approach, both the altitude and pitch control loops were substituted with RL agents. Finally, a single RL agent replaced both the altitude and pitch angle control loops, unifying control under a single agent. A comparative analysis has been made with the widely used conventional PID controls to assess the effectiveness of the each implemented control system. The RL controllers outperformed the baseline PID controllers, among which the unified RL controller achieved a steady-state error of 0.58 meters and a transient response time of 5 seconds, compared to the PID controller’s 1.11 meter steady-state error and transient response time of 16 seconds, thereby reducing the error by 48% and improving the response time by nearly 69%. These results demonstrate the superior accuracy and response efficiency of the proposed RL-based control strategies. Notably, the implementation featuring a single RL agent yields promising results, highlighting the capacity of RL agents to handle complex control challenges. This approach simplifies the control system design by eliminating the need for a multiple-loop architecture. The outcomes of this study underscore the potential of RL-based controllers to enhance the performance of UAVs. Furthermore, the results offer valuable insights for developing future UAV control systems, emphasizing the advantages of RL techniques over traditional PID controls.