Controller design via adaptive critic and model reference methods

Dynamic programming (DP) is a principled way to design optimal controllers for certain classes of nonlinear systems; unfortunately, DP is computationally very expensive. The reinforcement learning methods known as adaptive critics (AC) provide computationally feasible means for performing approximate dynamic programming (ADP). The term 'adaptive' in AC refers to the critic improved estimations of the value function used by DP. To apply DP, the user must craft a Utility function that embodies all the problem-specific design specifications/criteria. Model reference adaptive control methods have been successfully used in the control community to effect on-line redesign of a controller in response to variations in plant parameters, with the idea that the resulting closed loop system dynamics will mimic those of a reference model. The work (1) uses a reference model in ADP as the key information input to the Utility function, and (2) uses ADP off-line to design the desired controller Future work will extend this to on-line application. This method is demonstrated for a hypersonic shaped airplane called LoFLYTE/spl reg/; its handling characteristics are natively a little "hotter" than a pilot would desire. A control augmentation subsystem is designed using ADP to make the plane "feel like" a better behaved one, as specified by a reference model. The number of inputs to the successfully designed controller are among the largest seen in the literature to date.