Memory Efficient Exact and Approximate Functional Dependency Extraction with ParSIT

In the last decade, Big Data has been presenting more and more challenges to various fields of computer science that center around data processing. Functional dependency extraction, the process of finding rules and relationships between attributes of datasets, is one such application. In this paper, a new dependency extraction method is presented for finding both exact and approximate functional dependencies, that is also memory efficient for large datasets. The proposed method is a parallelized improvement of Sequential Indexing Tables. It is evaluated through benchmark datasets and analysis is given about its time and spatial complexity.