Blockchain-FRL for Vehicular Lane Changing: Toward Traffic, Data, and Training Safety

Reinforcement learning has been adopted to improve the efficiency of vehicular lane changing (LC) decisions. However, since the local vehicular data needs to be uploaded to the edge node for accomplishing the learning and decision tasks, the data and traffic safety are under critical threat. Illegal data usage or attacks can generate misleading LC decisions such as collisions or rollover, which severely degrades the traffic safety. Therefore, this article investigates a blockchain-federated reinforcement learning (FRL) approach for the LC decisions, which jointly accounts for the traffic, data, and training safety of the LC decision making. A vehicular reputation model based on the Bayesian risk situation is constructed and combined with the FRL. The proposed model can help evaluate the traffic safety, select the vehicles for participating in the FRL training, and adjust the FRL aggregation weights. The FRL can protect the data safety by enabling the exchange and aggregation of the LC decision network parameters. The blockchain can ensure the FRL training safety by recording the FRL task information. In addition, a Proof-of-Work (PoW) consensus scheme is devised to increase the FRL robustness, where the vehicles can collaboratively join the blockchain consensus and accomplish the FRL aggregation in a distributed manner. Two typical scenarios are selected for the experimental evaluation, including the highway scenario and the merging scenario. The experimental results indicate that the proposed method shows better efficiency, convergence, and message safety delivery ratio by comparing with the existing studies.