Towards In-Network Compact Representation: Mergeable Counting Bloom Filter Vis Cuckoo Scheduling

With the breakthrough of edge intelligence, we are witnessing a booming increase in distributed applications on edge nodes. These distributed applications need to apply a novel data representation algorithm to support data-information exchanging and data-information based decision among different edge nodes. As the most efficient data compact representation algorithm, Counting Bloom Filter (CBF) is an extension of Bloom filter, which enables updating data representation as well as inserting data into a representation. To facilitate distributed applications on edge nodes, edge nodes need to exchange and summarize the information of the data collected from different edge nodes. Impossible to merge with other CBFs, the existing CBF and its variants thus cannot be used for representing and exchanging data information among edge nodes. To handle this problem, we design a novel mergeable CBF, mergeCBF. Based on an insight about the counting processing of a CBF, we unfold the counter array of the conventional CBF to a group of bit arrays, and in order to support merging multiple filters, map each inputted item to the cells in this group of cuckoo-scheduled bit arrays instead of the counters in CBF. Experiments on real-world datasets demonstrate that mergeCBF can support conventional operations and merging operations in an efficient way without degrading the quality of the representation results.