Adaptive Dynamic Programming for Uncertain Linear Systems

This chapter presents a reinforcement learning‐inspired adaptive dynamic programming (ADP) approach for finding a new class of online adaptive optimal controllers for uncertain linear systems. Comparing the on‐policy and the off‐policy learning strategies, it can be seen that the former spreads the computational burden into different iteration time points, at the price of a longer learning process. The latter can achieve faster learning by making full use of the online measurements, at the expense of heavier computational efforts at a single iteration time point. The chapter uses two examples to validate, through numerical simulations, the effectiveness of the proposed algorithms. The first example is created using Simulink Version 8.5 (R2015a). It illustrates how the on‐policy learning can be applied to a third‐order linear system. The second example is implemented in MATLAB scripts. It applies the off‐policy learning strategy to design an approximate optimal feedback control policy for a diesel engine.