Abstract:
Decision-making for urban autonomous driving is challenging due to the stochastic nature of interactive traffic participants and the complexity of road structures. Althou...Show MoreMetadata
Abstract:
Decision-making for urban autonomous driving is challenging due to the stochastic nature of interactive traffic participants and the complexity of road structures. Although reinforcement learning (RL)-based decision-making schemes are promising to handle urban driving scenarios, they suffer from low sample efficiency and poor adaptability. In this paper, we propose the Scene-Rep Transformer to enhance RL decision-making capabilities through improved scene representation encoding and sequential predictive latent distillation. Specifically, a multi-stage Transformer (MST) encoder is constructed to model not only the interaction awareness between the ego vehicle and its neighbors but also intention awareness between the agents and their candidate routes. A sequential latent Transformer (SLT) with self-supervised learning objectives is employed to distill future predictive information into the latent scene representation, in order to reduce the exploration space and speed up training. The final decision-making module based on soft actor-critic (SAC) takes as input the refined latent scene representation from the Scene-Rep Transformer and generates decisions. The framework is validated in five challenging simulated urban scenarios with dense traffic, and its performance is manifested quantitatively by substantial improvements in data efficiency and performance in terms of success rate, safety, and efficiency. Qualitative results reveal that our framework is able to extract the intentions of neighbor agents, enabling better decision-making and more diversified driving behaviors.
Published in: IEEE Transactions on Intelligent Vehicles ( Volume: 9, Issue: 3, March 2024)
Funding Agency:
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Representation Learning ,
- Autonomous Vehicles ,
- Scene Representation ,
- Interactive ,
- Sampling Efficiency ,
- Latent Representation ,
- Self-supervised Learning ,
- Challenging Scenarios ,
- Urban Scenarios ,
- Road Structure ,
- Convolutional Neural Network ,
- Traffic Flow ,
- State Representation ,
- Markov Decision Process ,
- Latent Vector ,
- Methods In Scenarios ,
- Lane Change ,
- Global Prediction ,
- Left Turn ,
- Proximal Policy Optimization ,
- Auxiliary Task ,
- Multi-stage Structure ,
- Rule-based Model ,
- Self-supervised Learning Methods ,
- Future Horizon ,
- Average Success Rate ,
- Highest Success Rate ,
- Representation Learning Methods ,
- Graph Attention Network
- Author Keywords
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Representation Learning ,
- Autonomous Vehicles ,
- Scene Representation ,
- Interactive ,
- Sampling Efficiency ,
- Latent Representation ,
- Self-supervised Learning ,
- Challenging Scenarios ,
- Urban Scenarios ,
- Road Structure ,
- Convolutional Neural Network ,
- Traffic Flow ,
- State Representation ,
- Markov Decision Process ,
- Latent Vector ,
- Methods In Scenarios ,
- Lane Change ,
- Global Prediction ,
- Left Turn ,
- Proximal Policy Optimization ,
- Auxiliary Task ,
- Multi-stage Structure ,
- Rule-based Model ,
- Self-supervised Learning Methods ,
- Future Horizon ,
- Average Success Rate ,
- Highest Success Rate ,
- Representation Learning Methods ,
- Graph Attention Network
- Author Keywords