Distributed Dynamic Resource Management and Pricing in the IoT Systems With Blockchain-as-a-Service and UAV-Enabled Mobile Edge Computing

In this article, we study the pricing and resource management in the Internet of Things (IoT) system with blockchain-as-a-service (BaaS) and mobile-edge computing (MEC). The BaaS model includes the cloud-based server to perform blockchain tasks and the set of peers to collect data from local IoT devices. The MEC model consists of the set of terrestrial and aerial base stations (BSs), i.e., unmanned aerial vehicles (UAVs), to forward the tasks of peers to the BaaS server. Each BS is also equipped with an MEC server to run some blockchain tasks. As the BSs can be privately owned or controlled by different operators, there is no information exchange among them. We show that the resource management and pricing in the BaaS-MEC system are modeled as a stochastic Stackelberg game with multiple leaders and incomplete information about actions of leaders/BSs and followers/peers. We formulate a novel hierarchical reinforcement learning (RL) algorithm for the decision makings of BSs and peers. We also develop an unsupervised hierarchical deep learning (HDL) algorithm that combines deep $Q$ -learning (DQL) for BSs with the Bayesian deep learning (BDL) for peers. We prove that the proposed algorithms converge to stable states in which the peers' actions are the best responses to optimal actions of BSs.