Agent at the Edge: Opportunity and Challenges of Video Streaming Analytics at the CDN Edge

To provide high-quality streaming services to end users, streaming analytics applications need to process massive volumes of data promptly. Such applications suffer from high network transmission costs for transferring logs to a stream processor (cloud or on-premises), archiving, and computing costs for timely log analysis due to the volume, variety, and velocity of log data. This is especially important in live streaming, where millions of users play video simultaneously and consume network resources that are technically limited. A Distributed log analytic system can help to deal with this huge amount of data located at different locations and change rapidly. The advent of rich resources at the edge has enabled data processing close to the data source in a geo-distributed setup. Pushing log analytics closer to data sources is an effective strategy to reduce resource bottlenecks for the stream processor. This paper explores the benefits and drawbacks of deploying agents to analyze distributed logs, aiming to enhance the quality of video playback. Where increasing network and client diversity at the edge adds complexity to the task of processing live streams to end users situated across various networks and geographic locations. Furthermore, it introduces a mechanism to provide an abstract overview of the current streaming ecosystem resulting in better QoE.