Efficient Collaborative Multi-Agent Driving via Cross-Attention and Concise Communication

Reinforcement learning has been shown to have great potential applications in autonomous driving. For collaborative driving scenarios, multi-agent reinforcement learning can be used to explore efficient and collaborative driving strategies. However, it still faces the challenge of non-stationary. Traditional methods focus on evaluating the similarities between the real state of the teammate and the modeled state. There is also the issue of partial observability. It can be addressed by establishing communication to share information with other surrounding agents. However, prior approaches overlook the efficient communication problem caused by unprocessed and redundant information. To tackle these two challenges, we propose an approach named Multi-Agent Collaboration via Cross-Attention and Communication (MACAC). MACAC leverages the agent’s local observations to analyze and capture environment and interaction information, while also incorporating teammate modeling through the exchange of concise state information via communication. In addition, to improve the learning process, we integrate the noisy advantage technique into MACAC to enhance the agent’s exploration capabilities. As a result, vehicles can effectively adapt to dynamic environments and exhibit efficient collaborative driving skills. In all, experiments conducted on an autonomous driving simulator demonstrate that our approach surpasses the performance of the baseline algorithms.