Research on Apriori Improvement Algorithm Based on MapReduce

With the rapid development of e-commerce, how to discover useful association rules from large-scale shopping data has become a hot research topic. Apriori algorithm is a classical algorithm in the field of association rule mining, but it has the problem of low efficiency in large-scale data mining. In order to improve the execution efficiency of Apriori algorithm, this paper proposes an improved Apriori algorithm based on MapReduce by combining the distributed processing characteristics of MapReduce. The algorithm reduces the number of self-join comparisons of the algorithm by eliminating the self-join process of k-order candidate sets and instead using (k-1)-item candidate sets to join with frequent 1-item sets to generate new k-item candidate sets. Meanwhile, this paper implements the algorithm using Hadoop distributed computing framework for large-scale data experimental validation. The experimental results show that compared with the traditional Apriori algorithm, the improved MapReduce-based Apriori algorithm has higher execution efficiency and better scalability, and can be effectively applied to large-scale data mining scenarios.