Unsupervised Reinforcement Learning for Multi-Task Autonomous Driving: Expanding Skills and Cultivating Curiosity

In recent years, reinforcement learning (RL) has been widely used in decision-making. However, it still faces challenges when it is applied to autonomous driving, especially in complex multi-task scenarios. This paper introduces an unsupervised reinforcement learning(URL), called an improved Contrastive Intrinsic Control (CIC), to address this problem. CIC generates skills as transferable factors between different tasks to enable multi-task expansion. By comparing skills as potential state transfers with real state transfers, the mutual information between the two serves as the curiosity that drives the agents to explore the environment and gather experience in advance. This helps the collection of valuable experiences and the acquisition of effective skills. In the multi-task expansion phase, the unified training skills are used as a prior to enabling rapid convergence in various environments. Unified training is performed without rewards, followed by repeated training on multiple downstream tasks. Experiments are conducted in a highway environment, where three different driving modes are differentiated as separate RL tasks through reward functions. The experimental results demonstrate that the proposed method possesses the ability of multi-task learning. Compared to the Deep Deterministic Policy Gradient (DDPG) baseline, it achieves a 30% to 50% improvement in convergence speed at the single-task level and a 20% to 40% improvement in the final learning performance. Furthermore, even in complex tasks, where other RL methods struggle to learn effectively, it still achieves an obvious learning ability. This approach realizes an effective combination of curiosity mechanism, and RL decision making in the multitasking domain.