Optimizing distributed data structures using application-specific network interface software

Network interfaces that contain a programmable processor offer much flexibility, which so far has mainly been used to optimize message passing libraries. We show that high performance gains can be achieved by implementing support for application-specific shared data structures on the network interface processors. As a case study, we have implemented shared transposition tables on a Myrinet network, using customized software that runs partly on the network processor and partly on the host. The customized software greatly reduces the overhead of interactions between the network interface and the host. Also, the software exploits application semantics to obtain a simple and efficient communication protocol. Performance measurements indicate that applications that run application-specific code on the network interface are up to 2.5 times as fast as those that use generic message-passing software.