Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles

Autonomous Vehicles (AVs) have attracted significant attention in recent years and Reinforcement Learning (RL) shows remarkable performance in improving the autonomy of vehicles. In that regard, the widely-adopted Model-Free RL (MFRL) promises to solve decision-making tasks in connected AVs (CAVs), contingent on the readiness of a significant amount of data samples for training. Nevertheless, it might be infeasible in practice and possibly leads to learning instability. In contrast, Model-Based RL (MBRL) manifests itself in sample-efficient learning, but the asymptotic performance of MBRL might lag behind the state-of-the-art MFRL algorithms. Furthermore, most studies of AVs are limited the decision-making of a single AV only, thus underscoring the performance due to the absence of communications. In this study, we try to address the decision-making problem of multiple CAVs with limited communications, and propose a decentralized Multi-Agent Probabilistic Ensembles with Trajectory Sampling algorithm MA-PETS. In particular, in order to better capture the uncertainty of the unknown environment, MA-PETS leverages Probabilistic Ensemble (PE) neural networks to learn from communicated samples among neighboring CAVs. Afterwards, MA-PETS capably develops Trajectory Sampling (TS)-based model-predictive control for decision-making. Finally we empirically demonstrate the superiority of MA-PETS in terms of the sample-efficiency comparable to MFBL, and also validate the contributing impact of communications to multi-agent MBRL.