Adaptive Disorder Control in Continuous Data Streams

A disorder control is the key factor regarding accuracy and latency of query results when processing sliding window aggregates over continuous data streams. Many stream systems maintain buffers or leverage punctuations for the control. However, current systems suffer from the lack of adaptivity in the measure for estimating buffer sizes or punctuations, which may lead to inaccurate or delayed query results. To address this problem, we propose a probabilistic approach to using an adaptive measure derived from the distributions of tuple generation intervals and network latencies. In our approach, the measure estimates buffer sizes or punctuations according to a drop ratio, which denotes a percentage of tuple drops permissible in run-time processing. The drop ratio can be defined declaratively in a query specification and it provides a way for users to control the tuple drops as their intention. The experimental results show that our adaptive measure estimates more appropriate buffer sizes than ad hoc measures, which means that the proposed measure provides a lower latency, while retaining accuracy by satisfying the given drop ratio.