A Distributed Algorithm Based on Competitive Neural Network for Mining Frequent Patterns

Although FP-growth method is efficient and scalable for mining both long and short frequent patterns, and is about an order of magnitude faster than the Apriori algorithm, it is unrealistic to construct memory-based FP-tree when dataset is huge, because the FP-tree is too great to be held in memory entirely. In this study, we propose a novel method named Competitive-Network-based FP-growth method (CNFP), which combines competitive neural network with FP-growth to mine frequent patterns. In competitive learning, similar patterns are grouped by the network and represented by a single neuron. This grouping is done automatically based on data correlations. Huge database is divided into sets of similar data. After competitive learning, neurons in competitive layer are regarded as root to construct FP-sub-trees, in which transactions are similar to each other. Frequent patterns are mined based on FP-sub-tree to decompose the mining task into a set of smaller tasks, which dramatically reduces the search space. CNFP can mine frequent patterns on web log files and discover association rules between URL pages users access. Not only can it help us to discover the user access patterns effectively, but to provide the valid decision-making for the Web master to devise the personalized web site. Our experiments on a large real data set show that the approach is efficient and practical for mining association rules on website pages.