Quantifying the cluster speedup behavior in the realm of internode communication

Networked clusters of computers are commonly used to either process multiple sequential jobs concurrently, or to execute complex scientific and commercial parallel applications that are based on a message passing paradigm. In some areas, these clusters represent a cost-effective alternative to the more expensive supercomputers that are in use today. However, workstation clusters normally lack the high-powered interconnect fabric, as well as the more optimized protocol system architecture that is an integral part of commercial supercomputer systems. This study quantifies the speedup of parallel applications in a cluster environment in an analytical model that takes parallel processing, internode communication, and I/O latency into consideration. The model illustrates the (relative) impact that CPU and I/O parallelism has on the speedup behavior based on the 3 different inter-node communication scenarios: broadcast, nearest-neighbor, and request-response. This work revealed that it is important to consider the performance impact that the application has with regard to features such as remote memory access, completion notification, and address translation issues.