Latency-Aware Microservice Deployment for Edge AI Enabled Video Analytics

Video analytics plays a pivotal role in public safety (e.g., criminal suspect detection, traffic flow count, and illegal parking management), which assists the polices in monitoring all anomalous events in the street. In this paper, we consider the scenario with multiple video analytics applications from a single video stream. However, traditional monolithic architecture based video analytics applications shall seriously increase the response latency due to the resource contention of repetitive components. Therefore, we utilize the microservice architecture based video analytics (MAVA) to share the universal microser-vices in different applications, which shall decrease the response latency by reducing the computation load and increasing the resource utilization. To further achieve fast and accurate video analytics, the video analytics microservices are deployed in the edge closing to the cameras and users, and artificial intelligence (AI) methods are used in the microservices to realize specified functions. Therefore, an edge AI enabled MAVA (EAI-MAVA) architecture is proposed to achieve accurate video analytics in real-time. Furthermore, we formulate a microservice deployment problem to determine the location of each microservice in EAI-MAVA, which minimizes the response latency of all applications by considering the resource demands of microservices and the resource constraints of heterogeneous edge devices. Finally, a greedy-based heuristic algorithm is proposed to solve the non-convex microservice deployment problem, which obtains a sub-optimal solution with small loss of accuracy and reduces the solution time obviously.