Message proxies for efficient, protected communication on SMP clusters

This research addresses the problem of providing efficient, protected communication in an SMP cluster without incurring the overhead of system calls or the cost of custom hardware. It analyzes an approach that uses an idle SMP processor to run a message proxy, a communication process that provides protected access to the network. We implement message proxy based communication between a pair of IBM Model G30 SMPs and analyze the resulting overheads. We derive a performance model that shows that cache-miss latency within an SMP influences message proxy performance significantly. Simulations of a suite of ten parallel applications demonstrate that message proxies match the performance of custom hardware for three of the ten applications, and are between 10-30% slower for the other seven applications. A direct cache-update mechanism to reduce cache misses improves the performance of message proxies on communication-intensive programs by 7-25%. We conclude that message proxies provide a viable alternative to custom hardware for protected communication.