Joint Mobile Energy Replenishment and Data Gathering in Wireless Sensor Networks via Federated Deep Reinforcement Learning

Recent years have witnessed the proliferation of wireless energy transfer for Wireless Sensor Networks (WSNs), which are mainly used for data gathering in real-world applications. A number of studies have investigated mobile vehicle scheduling to charge sensor nodes via wireless Mobile Chargers (MCs). Unfortunately, most of them cannot parallelly charge all nodes in an intelligent manner with the global network attributes. Furthermore, the time-variable charging ignores the optimal data gathering, resulting in poor Joint Energy Replenishment and Data Gathering (JERDG). To fill this gap, this paper proposes a Federated Deep Reinforcement Learning (FDRL)-based JERDG (FERG) solution for WSNs. To this end, FERG first partitions the networks into a set of clusters to distribute the workload evenly among multiple MCs, and then designs an FDRL-based framework that incorporates various time-variant network attributes to determine the optimal schedule for charging and data gathering via multiple MCs and a base station (BS). The BS as the cloud server is responsible for global training of JERDG models, while multiple MCs will parallelly train local models to jointly charge energy-exhausted nodes and gather the data from all nodes in clusters. To reserve more personalized characteristics of each cluster, a density-based partial aggregation strategy is designed to train the global model. Furthermore, a reward-weighted update and selection solution is proposed to generate and exploit reference samples with high rewards. Simulation results obtained from various scenarios demonstrate that FERG significantly outperforms the state-of-the-art approaches in terms of network lifetime, energy efficiency and data collection latency.