Adaptive Policy Learning for Connected Autonomous Vehicles Defending Malicious Access Requests by Graph Reinforcement Learning

Access requests are used in cooperative tasks among Connected and Automated Vehicles (CAVs). Authorization decisions are determined by dynamic trust values in Trust Value-based Access Control (TBAC). Manual policy management in TBAC faces challenges when dealing with massive amounts of access data and dynamic environments. Traditional machine learning methods fail to adequately represent the complex spatio-temporal, decentralized and independent access requests. VPolicy-GRL (Policy Learning for Connected Autonomous Vehicles by Graph Reinforcement Learning) is proposed, enhancing the conventional Deep Q Network (DQN) with a graph neural network (GNN) framework. Firstly, a novel three-layer framework of TBAC for CAVs is proposed. Secondly, trust relations among CAVs are represented as several spatio-temporal dynamic graphs, enabling context-aware authorization decisions. Malicious access requests are categorized into unauthorized access requests caused by Sybil attacks and interfering access requests caused by DoS attacks. Thirdly, the DQN model for VPolicy-GRL is designed. Results demonstrate that VPolicy-GRL achieves superior feature abstraction with STGCN compared to GCN, which also obtains higher true rejection rates than Double DQN and DQN for both unauthorized and interfering access requests. Additionally, VPolicy-GRL exhibits improved convergence speed and stability.