ProbSky: Efficient Computation of Probabilistic Skyline Queries Over Distributed Data

Skyline queries have been widely used in various application domains including multi-criteria decision making, search pruning, and personalized recommendation systems. Given multiple criteria, skyline queries prune the search space of a large collection of multi-dimensional objects to a small set by returning objects that are not dominated by or superior to others. As an extension of the traditional skyline queries, probabilistic skyline queries aim to cope with uncertain datasets. This paper presents a novel MapReduce-based framework, ProbSky, in support of fast parallel distributed evaluation of probabilistic skyline queries on large high-dimensional data. ProbSky is adept at efficiently evaluating exact
$p$
-skyline queries on large uncertain data without compromising the quality of query results. From the theoretical point of view, we formally prove two pruning lemmas integrated with ProbSky to strengthen the early pruning capacity. ProbSky builds on top of three optimization techniques: dominant instance pruning, slab-based partitioning, and reference point-based acceleration. These extensive experiments driven by both real and synthetic datasets, reveal that compared to the state-of-the-art methods ProbSky speeds up the evaluation of the exact
$p$
-skyline queries on large high-dimensional data by at least one order of magnitude in most cases. Our experimental results also validate that by balancing the memory consumption and execution time among machines, ProbSky is adroit at curbing the bottleneck effect that causes severe system performance deterioration.