Fusion Control Tracking Strategy for Autonomous Vehicles: A Fast PPO Reinforcement Learning Based on Attention Mechanism and Physical Information

Accurate path tracking control is crucial for the performance of autonomous vehicle (AV). However, a single control algorithm struggles to adapt to complex dynamic scenarios with varying error states and path curvatures, leading to challenges in slow convergence and poor tracking performance. To address this, this paper proposes an adaptive fusion control scheme termed AMPIPPO that integrates an attention mechanism and a physical information model into Proximal Policy Optimization (PPO). First, FLnSM and NMPC are combined through PPO by balancing the two controllers. Then, the attention mechanism is incorporated to utilize error information and path curvature as state inputs for generating action outputs, significantly enhancing the adaptability of PPO. Additionally, physical information (PI) is implemented to strengthen both model-driven and data-driven approaches, achieving an accurate model estimation. Furthermore, this paper demonstrates the convergence of policy iterations in the AMPIPPO algorithm. Finally, a control scheme FLnSM-NMPC-AMPIPPO is proposed to achieve accurate tracking and enhance the adaptability of AV in dynamic scenarios. Compared to single controllers, the FLnSM-NMPC-AMPIPPO method exhibits higher tracking accuracy and faster convergence. Compared to other RL, AMPIPPO demonstrates superior performance in sparse data scenarios and significantly enhances AV prediction accuracy. The proposed FLnSM-NMPC-AMPIPPO demonstrates excellent performance through numerical simulations and experiments.