Online Distance Measurement for Tree Data Event Streams

Distances for trees have already been researched for decades. Thus most of the research is focused on static tree data. E.g. the calculation of XML documents when considering the aggregation of different data sources. Nowadays rising amounts of data require analysis on data streams. This results in the analysis of dynamic trees. Analysis on data streams require one-pass algorithms as well as efficient resource utilisation regarding memory and CPU. Our use case is the network traffic monitoring and analysis of batch jobs in a batch system. The monitoring data is delivered via a monitoring event stream per batch job. Each batch job implements specific workflows that build up the dynamic tree structure. To compare the batch jobs efficiently against each other over time, we utilise an incremental distance measurement for dynamic trees. This paper presents the incremental distance measurement. It utilises a sketching of trees using signatures. Furthermore we rely on hash tables to realise fast access and calculations. We analyse the properties of our incremental distance measurement and compare it to the well-known tree edit distance. All experiments are performed on real world data and confirm the feasibility and scalability of our approach.