An Extended Rough Set Model of Graph Structure for Incomplete Information System Using Spark

In the process of data cleaning, there are problems of high difficulty and low efficiency for processing IIS (incomplete information system) that has large size data. As this problem, distributed IMR (DIMR) is a method with data preprocessing capability for distributed IIS. We propose a distributed extended rough set of graph structure (DRSG) based on DIMR using Spark to accelerate parallel processing of IIS under big data. Specifically, we introduce the concepts of reliable element and controversial element, and establish a connection through non-symmetric similarity relation. Then, a graph structure with hierarchical relationship is proposed. And the path from the reliable element to the controversial element replaces the block in DIMR to calculate the approximation set. Finally, a parallel algorithm is proposed and the experiments show that DRSG reduces the resources required for computing and improves the speed in Spark. A comparative analysis has been conducted comprehensively to assess the performance of the proposed algorithm with DIMR on 8 datasets. And the results show that DRSG has faster speed. And the dependency is similar to or higher than DIMR.