Energy-Efficient Computation Offloading Based on Multiagent Deep Reinforcement Learning for Industrial Internet of Things Systems

The term Industrial Internet of Things (IIoT) was created to describe a specific area of the Internet of Things (IoT) that integrates information and communication technologies (ICTs) like cloud/edge computing, wireless sensor/actuator networks, and connected objects to enable and accelerate the development of Industry 4.0. IIoT applications (e.g., smart manufacturing, remote control of industrial machinery, and critical system monitoring) have various levels of criticality and Quality-of-Service (QoS) requirements. However, the characteristics of data collected by interconnected devices complicate the task of guaranteeing the QoS requirements in terms of latency and reliability in addition to the huge amount of energy consumption. As a potential solution, edge computing offers additional powerful resources in the proximity of the IIoT devices. Hence, the required QoS can be achieved by offloading computation-intensive tasks to edge servers. Moreover, the offloading process needs to be optimized to take full advantage. Unlikely, conventional optimization methods are very complex to be applied in the IIoT context. To overcome this issue, we propose a computation offloading approach based on deep reinforcement learning (DRL) to minimize long-term energy consumption and maximize the number of tasks completed before their tolerant deadlines. We introduce a system with multiple agents to deal with the increasing dimension of the action space, where each IIoT device is represented by its own DRL model. The goal of the model is to maximize a flexible and long-term reward. In addition, the DRL models are trained in the cloud and make decisions online in the edge servers, allowing quick decision making by avoiding iterative online optimization procedures. The performance of the proposed approach is evaluated through simulation. The proposal shows promising results compared to other approaches.