Sustainable Federated Learning with Mobile Crowdsensing: A DRL Approach for Learning Efficiency Maximization

In this paper, we propose a Sustainable Sensing Federated Learning (S2FL) system where Internet-of-Things (IoT) devices and mobile users harvest energy wirelessly to perform data sensing, local Federated Learning (FL) training, and model update transmissions. We formulate a joint optimization problem that considers transmission power, CPU frequency, bandwidth allocation, and time allocation to maximize long-term learning efficiency. To solve this complex and dynamic problem, we develop an algorithm that integrates Deep Reinforcement Learning (DRL) with optimization techniques, leveraging the Deep Deterministic Policy Gradient (DDPG) algorithm for time allocation and a Lagrangian-Based Block Coordinate Descent (BCD) Method for per-slot resource optimization. Simulation results demonstrate that our proposed DDPG-based DRL-S2FL algorithm significantly outperforms benchmark schemes, achieving up to 15 % higher average reward compared to Deep Q-Networks (DQN) and 40 % greater performance than Random strategies. This work highlights the effectiveness of combining advanced optimization techniques with deep reinforcement learning to enhance federated learning performance in dynamic wireless environments.