Intelligent Live Video Streaming for Object Detection

These days, sensors and cameras are being deployed on an increasingly large scale. Furthermore, the rapid development of machine learning models for computer vision now presents novel opportunities for the use of artificial intelligence (AI) and Internet of Things (IoT) combinations in various application scenarios. However, challenges remain in supporting low-latency video streaming from distributed mobile IoT devices under dynamic network environments, and overcoming video data quality degradation that results from weather “noise”, which reduces the accuracy of AI-based data analyses such as object detection. In this paper, we propose a live video stream processing system for supporting intelligent services that integrates the following features. First, to cope with dynamic networks and achieve low latency, our approach employs a peer-to-peer (P2P)-based virtual network at the edge and a multi-tiered architecture composed of IoT cameras, edge, and cloud servers. Second, we construct a flexible messaging system for video analysis built upon SINETStream, which is a messaging system that adopts a topic-based pub/sub model. Third, we implement a framework that can remove weather-related (rain, snow, and fog) noise by applying weather classification and adaptive noise removal models that improve the accuracy of video analysis from data collected outdoors. The latency, throughput, and image quality benchmark experiments conducted to validate the feasibility of our proposed system showed that the process resulted in image quality improvements of approximately 30% (on average).