Deep Reinforcement Learning of Position and Velocity PID Control for Rotational Wing Unmanned Aerial Vehicles

A Deep Reinforcement Learning (DRL) network in a mission controller was engaged with position and velocity Proportional, Integral, and Derivative (PID) flight controllers in a Rotational Wing Unmanned Aerial Vehicle (RWUAV) as a Software In The Loop (SITL). The DRL network was trained with 50-point flight telemetry data including gyroscope at 10Hz rate, and it dynamically updated PID controllers with their coefficients to hold a target position as initial climb and final approach flight phases, where a RWUAV is susceptible to most accidents. The DRL network was implemented with 5 inner layers and REINFORCE TF-agent. As a baseline, a static PID RWUAV's position holding response under random wind direction and velocity was examined. With mean velocity of 5m/s random wind, the DRL-engaged RWUAV's training and evaluation processes were collected and compared with static PID controller RWUAV performance. The DRL network with dynamic position and velocity PID controller coefficient updates improved position hold consistency by 65% at 0.17m on average from a static PID controller's 0.50m under 5m/s average velocity random wind.