UAV-Assisted Wireless Cooperative Communication and Coded Caching: A Multiagent Two-Timescale DRL Approach

In emergency scenarios, strong mobility and serious interference cause unstable transmission of on-site information such as close-up photos and high resolution videos, which requires a robust temporary communication network. In this paper, we focus on a UAV-assisted wireless cooperative communication and coded caching network, where emergency command vehicles and a UAV serve as content providers (CPs) to cache and transmit coded fragments or complete files for rescuers regarded as content requesters (CRs). The delivery success probability and content hit ratio are theoretically derived by incorporating the physical connectivity and social relationship between CPs and CRs. Aiming at maximizing the overall content hit ratio, we propose a multiagent two-timescale deep reinforcement learning (MA2T-DRL) algorithm to jointly optimize the transmission power and caching strategies for CPs. Specifically, we develop a two tier deep-Q networks (DQNs) framework integrating a slow-timescale DQN (ST-DQN) and a fast-timescale DQN (FT-DQN) for caching decision-making and power decision-making respectively, and then the QMIX framework is leveraged to aggregate all the outputs from local ST-DQNs. Considering the cooperative characteristics of coded caching, we further propose a novel clustering method for CPs such that CPs in the same cluster have the same willingness to serve CRs, and each cluster is regarded as the agent for training which further reduces the aggregation scale of the mixing network. Simulation results show that the proposed MA2T-DRL algorithm is efficient in model training, and presents the advantages in performance and complexity compared with the single-agent centralized training and the multiagent independent distributed training.