Resource Allocation for UAV-Enabled Spatio-Temporal Crowdsourcing in Smart Cities

Uncrewed aerial vehicle aerial vehicle (UAV)-enabled spatio-temporal crowdsourcing has a promising potential for efficient, real-time data collection from Internet of Things (IoT) in smart city applications. However, the existing works for such a problem mainly adapt to simple ideal synthetic scenarios and neglect the characteristics of the urban environment. In this paper, we consider both the system characteristics and urban environment and formulate the problem using both mathematical formulation and the Markov decision process. To ensure the timeliness and fairness of data collection from widely distributed IoT devices, we use the age of information (AoI) to represent the freshness of data. Such a problem jointly minimizes the average AoI and energy consumption of UAVs and IoT devices under the constraints of time sensitivity, regional boundaries, no-fly zones, UAV dynamics, and transmission range. A deep reinforcement learning (DRL)-based UAV-enabled spatio-temporal framework is proposed where three state-of-the-art DRL algorithms are introduced, namely twin delayed deep deterministic policy gradients (TD3), proximal policy optimization (PPO), and soft actor-critic (SAC). They all approximate the policy and value functions by deep neural networks based on actor-critic architecture. Experiments are conducted in both the synthetic and the real-city scenarios with real urban building data, and the results demonstrate the proposed framework's effectiveness, scalability, and transferability. Moreover, the proposed TD3, PPO, and SAC outperform deep deterministic policy gradient algorithm, random strategy, and meta-heuristic algorithms.