A Reinforcement Learning Based Service Scheduling Algorithm for Internet of Drones

Originally invented by the military for warfighting, drones have broken through adamant barriers established by traditional commercial and civilian industry and are quickly becoming an accepted part of mainstream. In order to enable drone technology to reach its full potential and integrate heterogeneous drones into existing workflows, Internet of Drones (IoD) has been proposed as a future aerial-ground communication architecture, where drones frequently contact Zone Service Providers (ZSPs) for up-to-date information. When many drones intend to access data through a ZSP concurrently, service scheduling plays a significant role in improving data accessibility. In practice, however, the limited bandwidth and coverage range of ZSP and the high speed of drones make the problem of service scheduling challenging. In this paper, we propose a reinforcement learning based service scheduling algorithm, also called RELESS, to optimally satisfy the service requests of drones in the IoD. In RELESS, the interaction between the ZSP and drones is formulated as a Markov decision process (MDP) which will be solved by the Q-learning algorithm to produce an optimal service scheduling policy. During this process, the ZSP adopts an ∊-greedy exploration method to continuously fine-tune its service scheduling policy with various system states, which is guaranteed to converge to an optimal policy. We develop a discrete-event driven simulation framework using OMNeT++, implement RELESS and its counterparts, and conduct simulation experiments for performance evaluation and comparison. Numerical results demonstrate that RELESS can improve service request satisfaction ratio, service request satisfaction latency, as well as data size satisfaction ratio, indicating a superior service scheduling approach in the IoD.