Parallel pull-based LRU: a request distribution algorithm for clustered Web caches using a DSM for memory mapped networks

The SIRAC laboratory has developed SciFS, a distributed shared memory (DSM) that tries to benefit from the high performance and the remote addressing capabilities of the scalable coherent interface (SCI) memory mapped network. We use SciFS for high performance cluster computing and we also experiment with it to build large scale clustered Web caches. We propose Whoops! a clustered Web cache prototype with a new request distribution algorithm, called PPBL (Parallel Pull-Based LRU), especially designed for use with a memory mapped network and a DSM system. Unlike other distribution algorithms the decision is distributed over all nodes thus providing better scalability. We evaluate a first PPBL implementation and discuss scalability issues. Then, we propose a solution to build a scalable implementation of PPBL. We conclude with other improvements that can be achieved to build efficient large scale clustered Web caches using a DSM over memory mapped networks.