Vertical-Intersection-Based Top-Down Algorithm for Frequent Itemset Mining on MapReduce

Frequent itemset mining (FIM) is a critical problem in association rule mining(ARM), sequence mining, and biomedical text summarization. Apriori-like parallel FIM algorithms scan a database multiple times and generate an excessive number of candidate itemsets for counting their frequencies. It suffers from potential problems such as high I/O traffic and a huge amount of memory usage. Dist-Eclat-like parallel FIM algorithms address the scalability problem. However, they are incapable of constructing an in-memory vertical form to accommodate a large-scale database. This paper proposes a vertical-intersection-based top-down algorithm for frequent itemset mining on MapReduce. The concept of transaction subcube is introduced as all the closed itemsets and their frequencies are derived by employing the vertical intersection method. The frequently closed itemsets are translated into frequent itemsets quickly in the top-down traveling manner with the aid of the negative form of subcubes. Thus, the proposed vertical-intersection-based algorithms have the capability of highly parallel processing, resulting in speeding up the system performance significantly.