SVDE: Serverless Framework for Low-Latency Video Analytic Queries With Hardware Disaggregation

Video analytics applications are gaining popularity among serverless environments. One issue that appears in existing serverless platforms is that they do not fully exploit opportunities to efficiently handle video chunking as they assume that all video chunks display similar computation and communication overheads. Those overheads vary and benefit from fine-grained handling. Scheduling non-uniform chunks is even more challenging when the hardware resources are heterogeneous and the network between the resources is non-uniform. To address these challenges, we propose SVDE, a heterogeneous serverless cloud framework for massive video processing workloads. SVDE employs a trained decision tree regression to efficiently decide where to process each video chunk, by holistically considering the non-uniform chunk sizes, heterogeneity of the node hardware, queuing status of each node, and an unbalanced network. Furthermore, we develop an efficient operator backend that will be open‐sourced as part of SVDE Ċompared to prior works, SVDE achieves up to 3.2$ \times $ speedup on ten real-world video workloads due to its holistic scheduling decision-making, while our operator backend outperforms the popular Pytorch JIT backend by 5$ \times $.