Software Orchestrated and Hardware Accelerated Artificial Intelligence: Toward Low Latency Edge Computing

Driven by the expeditious wireless evolution and growing complexity of Internet of Things systems, edge intelligence has been widely recognized as a novel paradigm to enable ubiquitous smart industry and consumer applications, which uses mobile edge computing (MEC) to push artificial intelligence (AI) related computing to the network edge near mobile terminals and data sources for low-latency data processing. However, design and deployment of edge intelligence remain extremely challenging due to resource-constrained edge computing environments and latency-related considerations. These motivate us to explore edge intelligence from different perspectives, particularly software orchestration (e.g., AI model design, model optimization, and resource management) and hardware acceleration methods (e.g., AI chip customization). However, the performance improvement of edge intelligence using just one single perspective is extremely limited, especially when its advantages are exhausted, which seriously hinders further performance development. In this article, we propose an edge computing framework with co-acceleration of software orchestration and hardware to break through the bottleneck of a single acceleration perspective on performance improvement, enabling high-quality edge intelligence service deployment. Simulation results show that the proposed edge computing framework has dramatically improved system performance in facilitating low-latency edge intelligence.