SHIFT: Multi-agent Reinforcement Learning for Spatiotemporal Mobile Traffic Shaping via Dynamic Pricing

Due to the rapid growth of mobile traffic, network operators have difficulty investing in sufficient wireless network equipment to meet the demand for mobile traffic, especially peak demand. To maximize investment efficiency, we consider dynamic pricing that allows operators to defer infrastructure investments by spreading out peak demand. In dynamic pricing, operators announce spatiotemporal prices in advance so that mobile users move back and forth in the spatial and temporal domains to reduce peak loads. To this end, we design a decentralized partially observable Markov decision process (Dec-POMDP) framework for spatiotemporal dynamic pricing with non-linear transitions of Boltzmann rationality and nonconvex objectives. To solve the Dec-POMDP problem with only local information, we introduce a model-free, value-based, multi-agent deep reinforcement learning (DRL) algorithm, termed SHIFT, that uses centralized training and decentralized execution (CTDE). Through evaluation using real traffic data from Telecom-Italia, we reveal that SHIFT outperforms other competitive schemes in supporting multiple agents in a scalable way and reducing peak demand.