MrFIM: A MapReduce Approach for Frequent Itemset Mining in Big Data

Nowadays simultaneous extraction algorithms for finding common element sets does not allows concurrent execution, load steadiness, data allocation, and recovery mechanism on huge clusters. Therefore, we propose a Simultaneous Common Elements Extraction algorithm (SCEE) using MapReduce. To accomplish reduce storage and to prevent constructing traditional pattern bases, SCEE algorithm integrates the frequent items ultrametric tree, instead of traditional FP-trees. In this algorithm, the number of MapReduce tasks are three, designed to finish the whole extracting job. The mappers of third MapReduce job individually perform the decomposition operation on element sets, while the reducer will complete the grouping process by building ultrametric trees, and extraction of these trees distinctly. We also implemented an additional MapReduce job to find the top-K items from these frequent itemsets, we called this MapReduce job as an Aggregate Function. We designed the algorithm on cluster of computers. We demonstrate that the SCEE algorithm on the cluster is subtle to data allocation and dimension, since element sets with dissimilar length have different decomposition and building expenses. The real-world data for extensive experimentations prove that our proposed technique is effective and extendable.