Joint Task Offloading and Energy Allocation for UAV-based Fog Computing Through Federated Deep Reinforcement Learning

Internet of Things (IoT) devices face significant challenges due to their limited capabilities and battery life especially in hazardous regions. Furthermore, the availability of telecommunication infrastructures in these areas is insufficient or event non-existent. As a result, transmitting the data collected by IoT devices becomes a formidable undertaking. Moreover, the limited lifespan of IoT devices, which rely on batteries for operation, exacerbates the situation. Recent research has explored the potential of unmanned aerial vehicles (UAVs) as effective means to facilitate air-ground communications and gather data from IoT devices deployed in harsh environments. However, the ability of UAVs to provide sustainable and reliable services is hindered by their dependence on limited battery power. Therefore, the UAV mobile edge/fog computing paradigm offers low latency communication, computation and storage capabilities for offloading intensive tasks of IoT devices. In addition, the utilization of current technology, such as Wireless Power Transfer (WPT), presents itself as a viable means to facilitate the capability of unmanned aerial vehicles (UAVs) to acquire power without the need for physical connection through ground base stations (GBS). Furthermore, this technology can also be employed to wirelessly replenish the energy of Internet of Things (IoT) devices situated in distant regions. In this study, we examine a scenario where UAVs providing offloading and energy services to UEs, making binary decisions for data offloading and continuous decisions for energy allocation. Using a federated deep reinforcement learning (FDRL) approach, we address privacy concerns and data leakage. The network simulation aims to determine the optimal number of UAVs a single fog node can serve and assesses the maximum UEs each UAV can handle. Our proposed framework’s performance is compared to single-agent and traditional central training, using mean reward as a metric. We evaluate mean delay over training iterations, comparing with central learning, and assess mean energy consumption against traditional central learning.