Performance evaluation of NSF application benchmarks on parallel systems

The National Science Foundation (NSF) recently released a set of application benchmarks that would be a key factor in selecting the next-generation high- performance computing environment. These benchmarks are designed to capture the salient attributes of those science and engineering applications placing the most stringent demands on the system to be provisioned. The application benchmarks consist of six codes that require large amount of memory and work with large data sets. In this work, we study the complexity, performance, and scalability of these codes on four machines: a 512-processor SGI Altix 3700, a 512-processor SGI Altix 3700/BX2, a 512-processor dual-core based SGI Altix 4700, and a 128-processor Cray Opteron cluster interconnected by the Myrinet network. We evaluated these codes for two different problem sizes using different numbers of processors. Our results show that per processor the SGI machines, using the Intel Itanium-2 processor, are faster than the Cray cluster, using the AMD Opteron processor, by a factor of up to three. Also, we found out that some of these codes scale up very well as we increase the number of processors while others scaled up poorly. In addition, one of the codes achieved about 2/3 of the peak rate of an SGI Altix processor. Moreover, the dual-core based system achieved comparable performance results to the single-core based system. Finally, we provide some limitations and concluding remarks.