Focused web crawling: A framework for crawling of country based financial data

Crawling the Web quickly and entirely is an expensive, unrealistic goal because of the required enormous amounts of hardware and network resources. But when only information about a predefined topic set is desired, a specialization of the aforementioned process called “focused crawling” is used. A focused crawler is an agent that targets a particular topic and visits and gathers only a relevant, narrow Web segment while trying not to waste resources on irrelevant material. Compared to the standard web search engines, focused crawlers yield good recall as well as good precision by restricting themselves to a limited domain. In this paper we will introduce the technique of focused crawling of country based financial data.