Geo-Distributed Analytical Streaming Architecture for IoT Platforms

The surge in real-time IoT data introduces scalability and computational challenges, necessitating advanced architectural and technological solutions. Streamed data processing is increasingly adopted across industries to enhance operational efficiency by extracting insights from vast, unstructured datasets. However, complex analytical tasks, such as multi-join queries, often require stateful iterative calculations on high-volume, high-velocity data, which challenges conventional programming models like MapReduce. This paper introduces an architectural model enabling application developers to create intricate streaming computational logic within an IoT platform. Our architecture supports scalable applications across distributed edge-tier nodes, particularly for iterative analytical operations on streamed data. We discuss core concepts and a timestamp model (borrowed from the timely data-flow concept) attached to data items circulating between computational blocks, which can execute concurrently on different edge-tier nodes. Additionally, we detail a buffer management mechanism that dynamically adjusts memory size in each computational block on nodes with limited capacity. This mechanism considers application performance requirements and runtime conditions to optimize buffer sizes. Performance evaluation against cloud-tier alternatives confirms the effectiveness of our solution. Experimental results show a significant reduction in p-99 delay compared to cloud-tier deployment with a database engine for analytical applications involving multi-join operations.