DDPG-based Hybrid Spectrum Sharing for AOI Guaranteeing of Power Services

To alleviate the scarcity of spectrum resources in smart grid communication networks (SGCN), dynamic spectrum sharing (DSS) has become an important solution. In this paper,we consider both licensed spectrum and shared spectrum and propose a hybrid spectrum sharing (HSS) mechanism that significantly extends the use range of spectrum. Taking into account the importance of information timeliness in SGCN, we introduce age of information(AOI) into the DSS scenario of smart grid for the first time and categorize power terminals into two types, AOI-tolerant power terminals (ATPTs) and AOI-sensitive power terminals (ASPTs), according to the demands of different power services. In order to satisfy the communication demands of different power terminals in the long term, the spectrum sharing problem oriented to the communication demands of power terminals is formulated as a Markov Decision Process(MDP) and a deep deterministic policy gradient (DDPG)-based HSS algorithm is proposed, which is able to keep the AOI of ASPTs at a low level while providing high average throughput for ATPTs. Simulation results show that compared to the existing algorithms, the DDPG-based HSS algorithm is able to reduce the AOI of ASPTs by an average of 19.87% and improve the throughput of ATPTs by an average of 64.11%.