DRL-Based Federated Learning for Efficient Vehicular Caching Management

In this study, we present a hybrid deep reinforcement learning (DRL) algorithm, trained using vehicular federated learning (VFL), specifically tailored for dynamic vehicular networks with historical data. Our approach utilizes VFL-based DRL to refine the caching scheme in these networks, focusing on predicting and storing the most effective content nearby to enhance cache efficiency and reduce content request delays. We propose a modified proximal policy optimization (mPPO)-based approach for the DRL-based decision making for caching management, which combines the advantages of proximal policy optimization (PPO) and double deep Q-network (DDQN). Our study encompasses a vehicular framework that includes a central edge node (CEN), roadside units (RSUs), unmanned aerial vehicles (UAVs), and vehicles equipped with historical data. We tackle the challenges posed by varying vehicle density and mobility, nonuniform RSU coverage, and constrained caching capacity. Through comprehensive simulations, we demonstrate that the mPPO outperforms the conventional DRL methods like PPO and DDQN, as well as heuristic approaches. These results underscore the efficacy of the VFL-based mPPO in dynamic vehicular networks, confirming its potential as a viable solution for real-world applications.