LLMCTS: Large Language Model Prior Guided MCTS for Autonomous Vehicle Decision-Making

Although the technique of combining Deep Reinforcement Learning (DRL) with Monte Carlo Tree Search (MCTS) has shown promise in Autonomous Vehicle (AV) decision-making, MCTS suffers from high computational resource consumption and low initial exploration efficiency due to its search characteristics. To address these issues, we propose LLMCTS, a framework that integrates large language model (LLM) prior knowledge into the MCTS algorithm. Specifically, LLMCTS first utilizes the expert reasoning of LLM to guide AV decision-making and stores it as policy data. Then, MCTS uses this data together with the policy generated by tree search for DRL training. Finally, MCTS outputs the AV decision. To verify the effectiveness of the LLMCTS framework, we conduct an experimental evaluation on an unprotected left-turn intersection. The results show that LLMCTS outperforms the baseline methods in all aspects. In particular, we also demonstrate that LLMCTS can guarantee the performance of MCTS with a small number of simulations.