UrbanEnQoSPlace: A Deep Reinforcement Learning Model for Service Placement of Real-Time Smart City IoT Applications

Multi-access Edge Computing (MEC) enables IoT applications to place their services in the edge servers of mobile networks, balancing Quality-of-Service (QoS) and energy-efficiency. Previous works consider compute requirements, while the IoT and latency/bandwidth per-flow communicate requirements are largely ignored. Moreover, the Smart City domain presents unique challenges – modeling the Urban Smart Things (USTs – urban IoT clients), their connectivity with MEC network, diverse resource requirements (compute, communicate, and IoT) of application services, modeling the federation of multiple MEC providers in a city, which we consider in this article. To address these research gaps, we propose: i)
UrbanEnQoSMDP
– formulation for energy and QoS (latency) optimized service placement for a set of applications in the ‘Urban IoT-Federated MEC-Cloud’ architecture to satisfy applications’ compute, per-flow communicate, and IoT requirements; ii)
‘’$\epsilon$ε-greedy with mask”
policy for apriori satisfaction of IoT requirements by shortlisting suitable USTs; iii)
UrbanEnQoSPlace
– proposed multi-action Deep Reinforcement Learning (DRL) model, designed from Dueling Deep-Q Network, that uses the proposed policy to solve the UrbanEnQoSMDP for simultaneously placing all services of an application. Extensive simulation results illustrate efficacy and scalability of proposed model against state-of-the-art DRL algorithms (better convergence, higher rewards, lesser runtime; proposed policy w.r.t fewer violations).