Resources allocation schemas for web information monitoring

The web is an extremely dynamic world where information is updated even every second. A web information monitoring system fetches information from the web continuously and finds changes by comparing two versions of the same page. The updating of a specific web page is modeled as a Poisson process with parameter to indicate the change frequency. As the amount of computing resources is limited, it is necessary to find some policies for reducing the overall change-detection time. Different allocation schemas are evaluated experimentally to find out which one is the most suitable for the web information monitoring problem. The experimental data shows the runtime characteristics of the overall system performance and the relationship to the total amount of resources.