Dynamic neural network control through fuzzy Q-learning algorithms

An efficient Q-learning paradigm implemented on a fuzzy CMAC network is proposed. The fuzzy CMAC network topological architecture is described. First, the continuous states of the system are partitioned into a number of fuzzy boxes. Second, the proposed fuzzy CMAC evaluates the Q-values of agents in the fired fuzzy boxes and chooses control actions with maximum Q-values. Then a critic generates an external reinforcement signal according to the outcome or the effect of the control at every time-step, which is used later for further improving the estimation of these Q-values. To speed up the convergence of reinforcement learning, the traditional PID controller with several groups of different parameters is adopted so as to collect a number of taught-lessons. These taught-lessons together with the experienced lessons generated automatically, are sequentially replayed and learned, respectively, under the guidance of different reinforcement mechanisms. The hybrid adaptive and learning control system is applied to the control of a pH-neutralization process. Simulation investigations show that the fuzzy connectionist Q-learning control system has more adaptive, higher intelligence, and stronger generalization ability compared to neural network or fuzzy neural network control techniques using supervised learning.