Deep Reinforcement Learning-Based UAV Base Station Deployment for Content Delivery in Cellular IoT Networks

Unmanned Aerial Vehicle base stations (UBSs) can be used to assist a cellular Internet of Things (IoT) network to provide content delivery service for ground users. This paper studies the UBS deployment problem in a cellular IoT network for content delivery and formulates the problem as a joint mixed-integer linear programming problem with an objective to minimize the average content delivery delay of all users in a service area in a time frame. The formulated problem is decomposed into three sub-problems: a content caching deployment problem, a UBS position deployment problem, and a BS association problem. A deep reinforcement learning-based UBS deployment (DRL-UD) algorithm is proposed to solve the problem. In the DRL-UD algorithm, an Informer-based user pattern prediction algorithm is introduced to predict the content request pattern and mobility pattern of users. Based on the prediction of user patterns, a two-layer proximal policy optimization (TLPPO)-based UBS deployment algorithm is introduced to solve the three sub-problems using a cache layer, a position layer, and an implicit enumeration method respectively. Simulation results show that the proposed DRL-UD algorithm can significantly reduce the average content delivery delay and increase the cache hit ratio of all users in the network.