DC-PPO for Joint User Association and Power Allocation in Dynamic Indoor Hybrid VLC/RF Networks

Hybrid visible light communication (VLC)/radio frequency (RF) networks have emerged as a promising next-generation networking technology that can provide high data rate and wide coverage while overcoming blockage problems of VLC and the limited available radio spectrum of RF. However, there has been limited research on the joint user association and power allocation problem in indoor hybrid VLC/RF networks with consideration of networking dynamics. To bridge the gap, we first formulate a joint user association and power allocation problem with the objective of maximizing the sum throughput of the hybrid networks with consideration of user mobility, time-varying blockages and interference between VLC access points (APs) and users, and practical capacity limitations of APs. Then we propose a distributed cooperative proximal policy optimization (DC-PPO) algorithm to solve the formulated mixed integer nonlinear programming problem, where each user is modeled as a hybrid agent that distributedly determines the user-AP association (discrete action) and the desired transmission power (continuous action) variables simultaneously based on local observations and limited information exchanged between neighboring nodes. The performance of our proposed method is compared with three baselines, including deep Q-network (DQN), max-power-max-signal-to-noise ratio (SNR), and random-power-max-SNR. Simulation results show that the proposed method is mobility and blockage-ware and outperforms the baseline solutions in terms of optimality, convergence, scalability, and robustness. Specifically, our proposed method can achieve $127 \%$ throughput improvement compared with that of DQN in 30mobile-user scenario.