By Topic

Analyzing the Web Crawler as a Feed Forward Engine for an Efficient Solution to the Search Problem in the Minimum Amount of Time through a Distributed Framework

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Qureshi, M.A. ; Dept. of Comput. Sci., Korea Adv. Inst. of Sci. & Technol., Daejeon, South Korea ; Younus, A. ; Rojas, F.

A web crawler forms the backbone of a search engine and this backbone needs a careful re- assessment that could enhance the efficiency of search engines. This paper conducts such a re- assessment from the perspective of systems and this is achieved through implementation and analysis of a web crawler "VisionerBOT" as a feed forward engine for search engines using the MapReduce distributed programming model. Our crawler implementations revisit the classical OS debate of threads vs. events, with a significant contribution from our work which concludes that events is the ideal way forward for web crawlers. Furthermore, in implementing the feed forward mechanisms within the web crawler, we came up with some important design considerations for the operating system research community which can lead to a whole new class of operating systems.

Published in:

Information Science and Applications (ICISA), 2010 International Conference on

Date of Conference:

21-23 April 2010