Evaluating Parallel Extensions to High Level Languages Using the HPC Challenge Benchmarks

Recent years have seen the development of many new parallel extensions to high level languages. However, there does not yet seem to have been a concentrated effort to quantify their performance or qualify their usability. Toward this end, we have used several parallel extensions to implement four of the high performance computing (HPC) Challenge benchmarks-FFT, HPL, RandomAccess, and STREAM-according to the Class 2 specifications. The parallel extensions used here include pMatlab, Star-P, and the official Parallel Computing Toolbox for MATLAB; pMatlab for Octave; and Star-P for Python. We have recorded performance results for the benchmarks using these extensions on the Ohio Supercomputing Center's supercomputer Glenn as well as several of the Department of Defense Supercomputing Resource Centers (DoD DSRCs). These results are compared to those of the original C benchmarks as run on Glenn. We also highlight some of the features of these parallel extensions, as well as those of gridMathematica for Mathematica and IPython for Python, which have not yet been fully benchmarked.