Piggybacking on UAV Package Delivery Systems to Simultaneously Provide Wireless Coverage: A Deep Reinforcement Learning-Based Trajectory Design

Various studies have explored the possibility of utilizing unmanned aerial vehicles (UAVs) as last-mile package delivery agents and aerial base stations in recent years. Despite the tremendous attention in these applications, nearly all of the studies assumed that the UAVs are serving just one purpose, but not both simultaneously; however, for a number of reasons, such as traffic congestion in the sky and energy & resource inefficiency problems, it seems more suitable for the UAVs to be serving several services concurrently. A few papers investigated the case in which the UAV acts as both a package delivery agent and a wireless transceiver, but in all of them, package delivery time constraints were never considered. Stemming from the observation that there are various ongoing industrial projects in UAV-based package delivery, we investigate the possibility of piggybacking on UAV-based package delivery infrastructures to also provide wireless coverage, and consider the problem of designing UAV trajectories that maximize the cumulative downlink sum rate of the ground communication users while simultaneously delivering packages under strict delivery time constraints. We use deep Q-learning (DQL) to solve this optimization problem, and demonstrate the successful formulation & implementation of our algorithm through several simulations.