An intelligent offline filtering agent for website analysis and content rating

The unregulated nature of the web means that anyone can make content available on the web, some of which could be harmful to children and unsuspecting adults. Content filtering is aimed at blocking out undesirable material from reaching the end user. Most existing software content filters make use an access control list which involves some sort of manual search, gathering and classification of undesirable web sites so that the software filter can block the access of these URLs. In this paper, we describe an Offline filtering Agent in terms of its two main modules: automated web page crawling and intelligent classification modules. Experimental results based on 1250 web pages are presented to show the effectiveness of our system. On the testing set, the agent was able to achieve a correct acceptance rate of 97% and a correct reject rate of 92.6%.