PreM-FedIoV: A Novel Federated Reinforcement Learning Framework for Predictive Maintenance in IoV

The Internet of Vehicles (IoV) enhances data availability by equipping a plethora of sensors, driving the automotive industry towards data-driven Predictive Maintenance (PreM) models. However, traditional centralized PreM solutions, requiring complete access to training data, raise concerns about data privacy. PreM in the automotive domain is more challenging than in many other fields, partly due to the varying distribution nature of data samples and the limited network connectivity time caused by vehicle mobility. To address these challenges, we propose the PreM-FedIoV framework, extending single-agent Double Deep Q-Network (DDQN) to Multi-Agent Double Deep Q-Network (MADDQN). In each round, each vehicle client uploads a data packet to the server based on the current contention window, containing its local model, local test Mean Absolute Error (MAE), and a timestamp. The server initially performs federated aggregation on the received local models. The MADDQN module then dynamically adjusts the contention window of each vehicle for the next round based on the local test MAE and communication statistical state, aiming to optimize communication costs and predictive performance. Additionally, we utilize NS-3 to create IoV simulations and deploy the PreM-FedIoV framework within NS3-gym. We choose Federated Averaging (FedAvg) and FedAdam following the IEEE 802.11p standard as baselines. The experiments demonstrate significant improvements in our framework compared to state-of-the-art algorithms. On the C-MAPSS dataset, we achieve reductions of up to 10.2% in MAE, 26.31% in average communication clock time per round, and 65.6% in the number of participating clients per round. For the Random Battery Usage dataset, with up to 4.55%, 24.44%, and 36.58% improvements in the respective metrics.