Portable petaFLOP/s programming: applying distributed computing methodology to the grid within a single machine room

According to today's best projections, petaFLOP/s computing platforms will combine deep memory hierarchies in both latency and bandwidth with a need for many-thousand-fold parallelism. Unless effective parallel programs are prepared in advance, much of the promise of the first year or two of operation for these systems may be lost. We introduce a candidate for a portable petaFLOP/s programming model that can enable these important early application programs to be developed while, at the same time, permitting these same applications to run efficiently on the most capable computing systems now available. An MPI-based model is portable, but its programming paradigm ignores the potential benefits of hardware support for shared memory within each network node. A threads-based model cannot directly cope with the distributed nature of the memory over the network. Therefore, a new, portable programming model is needed. The shared memory programming model dramatically simplifies the expression of dynamic load balancing strategies for irregular algorithms. The main strategy is a transparent self-scheduled task list performed in parallel so long as specified data-dependent conditions are met. The model used is a cluster of multiprocessor distributed shared memory machines with network-attached disks. Our experimental run-time system allows the programmer to view this computing platform as a single machine with a four-stage memory hierarchy, consisting of coherent processor cache, non-coherent local shared memory, global shared memory, plus a global disk file system.