Scaling Web sites through caching

A large jump in a Web site's traffic may indicate success, but this increased interest could turn into a nightmare if the site is not prepared to handle the bigger load. Site resources (processors, storage boxes, load balancers, LANs, and the like) could see their utilization rise to levels that generate increasingly long response times. Eventually, one of these resources - the so-called bottleneck resource - will reach 100 percent utilization, pushing the site's throughput to its maximum point. When a Web site becomes overloaded, customers grow frustrated with long waits and rejected requests. This situation can lead to an undesirable loss of site-generated revenue and may even tarnish the reputation of organizations relying on Web sites to support mission-critical applications. In this article, I discuss how caching technologies can improve Web site performance and scalability. I'll provide some simple quantitative expressions to let designers understand the most important trade-offs.