HeuRL: A Heuristically Initialized Reinforcement Learning Method for Autonomous Driving Control Task

Although reinforcement learning (RL) shows great intelligence in many simulation tasks, it hasn't been widely applied in real-world vehicle control tasks due to the simulation-to-real-world (Sim2Real) transferring difficulties. Vehicle models and road conditions in real world can be very different from those in simulators. As a result, the RL models trained by simulators usually fail and need to be trained again under a new environment, which is dangerous and time-consuming. Some hand-craft heuristic methods, by contrast, are independent of environmental characters and perform more reliably in an unfamiliar situation. In this paper, we introduce a heuristically initialized RL model (HeuRL), which sped up the learning convergence by 4 times and decreased the collisions by 90% during the training process under a new environment. The experiments were conducted in The Open Racing Car Simulator (TORCS), an open-source platform for real-time car racing simulation.