Learning with Dynamics: Autonomous Regulation of UAV Based Communication Networks with Dynamic UAV Crew

Unmanned Aerial Vehicle (UAV) based communication networks (UCNs) are a key component in future mobile networking. To handle the dynamic environments and UAV topologies in UCNs, reinforcement learning (RL) has been a promising solution attributed to its strong capability of adaptive decision-making free of the environment models. However, most existing RL-based research focus on control strategy design assuming a fixed set of UAVs, i.e., the number of UAVs does not change during the mission. Few works have investigated how UCNs should be adaptively regulated when the number of serving UAVs changes dynamically. This article discusses RL-based strategy design for adaptive UCN regulation given a dynamic number of UAVs, addressing both reactive strategies in general UCNs and proactive strategies in solar-powered UCNs. An overview of the UCN and the RL framework is first provided. Potential research directions with key challenges and possible solutions are then elaborated. Preliminary works are presented as case studies to inspire innovative ways to handle dynamic UAV crew with different RL algorithms.