A Multi-Agent Based Web Mining Model

A multi-agent based Web mining model is designed for the improvement of the efficiency of keywords based search engine. The model divides mining task into several parallel agents which coordinately work together, and the mining efficiency is improved greatly. Evolving from HITS, algorithm named Grabber in the model removes Link Farm pages in the expansion of root set, makes anchor text similarity calculation when crawling link page, and chooses pages by a brief conceptual analysis of page content. With the overcoming of the shortcomings of only text analysis or link analysis, Grabber enhances the search engine in understanding the user interest and crawling more Web pages to meet the needs of the users.