Online learning control law design based on policy gradient reinforcement learning

Morphing aircraft exhibit agile maneuverability within an extensive flight envelope, which imposes heightened requirements on the online adaptation capabilities of control laws and necessitates enhanced dynamic flight control performance. The coefficient freezing method frequently used in engineering applications entails individualized design procedures for each distinct operating point, a process that is both intricate and heavily reliant on human expertise. Moreover, once deployed, conventional flight control laws generally exhibit static performance with limited potential for online optimization and adjustment. This work presents an offline strategy gradient deep reinforcement learning approach to devise a self-learning architecture for the parameters of PI state feedback control laws. This architecture is trained automatically by traversing across multiple operational points of the morphing aircraft under control, thereby yielding a set of control laws adaptable to various operational conditions. Concurrently, the action network from this framework is extracted and implemented as an online learning module, enabling real-time adjustments to control law parameters during flight based on the observed control performance, thus augmenting the aircraft’s inherent online adaptive capabilities. The validity and effectiveness of the proposed methodology are substantiated through a case study focusing on tracking control in the pitch channel of a representative morphing aircraft.