Deep Reinforcement Learning With Fuzzy Feature Fusion for Cooperative Control in Traffic Light and Connected Autonomous Vehicles

A mixed traffic environment of manual driving and automatic driving will become the norm in future intelligent transportation systems. The deep reinforcement learning (DRL) method has shown significant promise in cooperative control for traffic lights and connected autonomous vehicles (CAV) in a mixed-traffic environment. However, the uncertainty and noise in integrating agents' observations can lead to inadequate exploration of environmental data by DRL algorithms. Consequently, these algorithms are prone to overfitting and becoming trapped in local optimal, which limits the performance of control strategies. To more effectively harness the gathered environmental data and thereby facilitate improved decision-making by agents, a DRL-based cooperative control method with fuzzy feature fusion (F3DRL) was proposed in this article. First, the adaptive fuzzy inference module is implemented to adaptively mitigate information uncertainty as the data from CAV is aggregated. Then, a deep information extraction module was introduced and integrated with the output of the adaptive fuzzy inference module to establish a parallel feature fusion module. The adaptive fuzzy inference module mitigates uncertainty in the extracted traffic environmental states, while the deep information extraction module facilitates the extraction of a more comprehensive environmental representation. The fusion of features derived from these two distinct modules aids DRL agents in making better action selections, which significantly enhances the effectiveness and stability of the F3DRL method. In simulations, F3DRL significantly reduced travel and delay times, fuel consumption, and CO$_{2}$ emissions, outperforming both traditional and state-of-the-art methods.