ParaNet: A Single Blocked Network for Mobile Edge Computing Devices

Nowadays, deep learning-based approaches have achieved significant performances in diverse applications. However, due to having millions of parameters and higher complexity, these high-performing architectures are not suitable to be deployed in edge devices, the Internet of Things (IoT), Vehicular edge computing, and microservices-based real-time applications. Though numerous approaches have proposed lightweight architectures to reduce required computational resources, there are still some concerns about latency, execution, and response time. To the best of our knowledge, no prior works have considered reorganizing the sequential blocks into parallel forward propagation i.e, converting sequential forward propagation into parallel forward propagation. In this paper, instead of reducing the time required by the network for end-to-end sequential execution, we propose a novel technique to obtain a parallel network called ParaNet to minimize the execution time by paralleling the network. Firstly, we dissect a CNN block-wise where all the blocks are deployed parallelly to construct ParaNet. Each block is treated as an individual network and can be deployed into different low computational edge devices for parallel processing. To further improve the performances we deploy the knowledge distillation technique into each ParaNet version. Our proposed method offers state-of-the-art results using low computational resources with very low execution delay compared to the corresponding baseline architectures. Our extensive analysis and results express the superiority of the ParaNet regarding both accuracy and execution time.

I. Introduction

In a few decades, deep learning-based techniques have drawn remarkable attention due to achieving gigantic performances in diverse tasks such as surveillance systems [2], [3], autonomous driving [4], [5], real-time computing in edge devices [6], [11] and so on. These complex architectures demand high computational resource devices because of having millions of parameters [7]. These requirements of high computational resources limit the scope of the deep learning-based methods in low computational resource-based IoT, edge, or mobile devices [8]. To improve this situation, a number of lightweight architectures have been proposed [9]–[11]. However, these lightweight architectures are less subject to reliable in terms of performance compared to heavy and complex architectures. Moreover, plenty of domain and task-oriented learning paradigms such as distributed learning [12], [13], federated learning [14], [15], decentralized learning [16], [17] etc., have been proposed to advance the scope of deep learning in edge computing, IoT, microservices and so on [18], [19]. Though these methods reduce the latency and required computational resources, there is still demand to minimize the execution and response time in real-time applications. To the best of our knowledge, no prior works focus on reducing the time required by the network for end-to-end sequential execution by paralleling the network through performing block-wise dissection.