End-to-End Autonomous Driving Algorithm Based on PPO and Its Implementation

With the commercialization of vehicles equipped with partial autonomous driving capabilities, achieving fully autonomous driving remains a hot topic among researchers. In contrast to modular autonomous vehicle control, end-to-end autonomous driving aims to replicate human driving by directly translating sensor inputs into vehicle control outputs, making it the ultimate goal in autonomous driving. In this study, we propose an end-to-end autonomous driving algorithm based on the Proximal Policy Optimization (PPO) reinforcement learning algorithm and verify its feasibility. Firstly, we construct an autonomous driving algorithm based on the PPO algorithm, where pre-encoded images and vehicle state information are input to a policy based on deep neural networks, directly generating speed and steering controls for the vehicle, thus achieving true end-to-end autonomous driving. Furthermore, using the CARLA simulation environment for autonomous vehicles, we define a reward mechanism for driving behavior and establish a reinforcement learning environment to train and validate the proposed algorithm. Finally, we analyze the algorithm's feasibility based on rewards, cumulative average rewards, average episode length, and average driving distance. While the algorithm still has a long way to go before practical application, it lays the foundation for further implementation of end-to-end autonomous driving algorithms.