Game-Theoretic Receding-Horizon Reinforcement Learning for Lateral Control of Autonomous Vehicles

Lateral control for autonomous vehicles (AVs) under uncertainties is an important research topic. Although there exist various control approaches, it is still a difficult problem to design a robust optimal controller for AVs under uncertainties in the conditions of high maneuverability and large curvature turns. This paper proposes a game-theoretic receding horizon reinforcement learning algorithm (GTRHRL) for the lateral tracking control of autonomous vehicles in special scenarios, such as high maneuvering conditions and large curvature turns, which improves control performance and enhances the robustness to uncertainties while possessing efficient online learning capabilities. The proposed learning-based control strategy combines both the advantages of receding horizon reinforcement learning and the differential game theory. Different from previous receding horizon reinforcement learning, the uncertainties imposed on the AVs are considered and formulated as a player by zero-sum differential games. In this way, the robustness of the controller is guaranteed. Meanwhile, we prove that the proposed control strategy can reach the Nash equilibrium as well as retaining the stability and optimality under uncertainties. Furthermore, the actor-critic algorithm including a critic neural network and two actor neural networks is designed to implement the control strategy. By Lyapunov stability theory, the stability of the implemented learning-based control strategy is analyzed, and the convergence analysis of the neural networks is performed. Various simulations are carried out to illustrate the superiority of the proposed learning-based control strategy. The experimental studies under special scenarios, such as high maneuvering conditions and large curvature turns are performed on an autonomous vehicle, and the results further validate the effectiveness and feasibility of the proposed strategy.