Q-Learning-Based Control for Discrete-Time Switched Affine Systems and Its Application to DC–DC Converter

In this paper, a new data-based Q-learning algorithm is proposed to address the optimal control issue for a class of discrete-time switched affine systems (SASs). The algorithm shifts the emphasis onto learning the optimal switching law directly from system input-output data, employing a neural-network-approximated Q-function as the key learning element. Firstly, the optimal control issue is transformed into solving the corresponding Bellman’s optimality equation based on the Q-function. Then, a new Q-learning algorithm is developed to find the optimal solution of system switching based entirely on the system input-output data, and a fully connected neural network is borrowed as the Q-function approximator. Moreover, considering the affine properties of SASs, the sequence of Q-functions generated remains bounded in proximity to the precise optimal solution. Finally, both the advantage and effectiveness of the proposed Q-learning based optimal control approach are verified by three examples, including a case study of DC-DC buck-boost converter.