Continual Reinforcement Learning for Autonomous Driving with Application on Velocity Control under Various Environment

Reinforcement learning based methods are extensively studied in autonomous driving. However, most existing methods, suffering from catastrophic forgetting, only work properly under certain scenarios. Therefore, this paper proposes a continual reinforcement learning approach to improve the control adaptability of autonomous driving systems. Specifically, the framework is based on a Soft Actor-Critic algorithm, which includes a shared feature extractor with regularization loss to learn task-specific output mapping. A linear multi-policy heads structure is designed to continuously learn different tasks without interference. Velocity control under various environment is taken as a case study, and three velocity control tasks with a high degree of overlap in the observation spaces are designed. In addition, multi-modal optimization objectives considering safety, efficiency and comfort for the three tasks are designed to evaluate the effectiveness of the proposed approach. The simulation studies are conducted in CARLA, which demonstrates the efficiency of the proposed method in terms of improving the continual learning capability of the model. Overall, the proposed continual reinforcement learning framework contributes to the development of adaptive autonomous driving systems to a large extent.