Design and performance of Maestro cluster network

Most clusters so far have used WAN or LAN-based network products for communication due to their market availability. However they do not always match communication patterns in clusters, thus incurring extra overhead. Based on our investigation for such overhead, we have optimized cluster communication at link layer. Partitioning each message in 16-byte packets, our optimization uses two techniques: (1) transferring in burst as many packets as the receiving buffer accepts at once, and (2) having each hardware component pass one packet to another in a pipelined manner. We have realized those two techniques in a link control hardware chip, referred to as MLC(Maestro Link Controller), and have constructed the Maestro cluster network using MLCs. This paper describes the feature of the Maestro cluster network and demonstrates the efficiency of our optimization techniques through performance experiments over this network.