EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless

Model inference in edge is often regarded as an effective method to alleviate high latency and enhance data privacy in edge-cloud collaborative computing environment. In this paper, we demonstrate that optimizing network communication in edge-cloud environment with limited bandwidth can enhance model inference performance. We first analyze network bottlenecks and the characteristics in model inference, then design a serverless inference system - EINS, to support collaborative optimization of network transmission and inference performance in edge-cloud environment. This system identifies concurrent network communication bottlenecks in multi-model deployment, dynamically scales capacity, and optimizes placement strategies and model transfer sequences. Real-world workload evaluation reveal that EINS can achieve a 5.7x throughput improvement and an average reduction of 62% latency in model instance startup.