An efficient asynchronous data transmission mechanism for data parallel languages

Reducing the overhead of data transmissions is crucial to harnessing the potential of distributed memory multiprocessors. Some optimization techniques such as overlapping communication and computation have been proposed. However, the most overlapping techniques are still limited by the following factors. (1) There is no linguistic support for overlapping communication and computation in data parallel languages; (2) There is no global analysis performed for array references over different nested loops, which will decide how to overlap communication and computation; (3). There is no formalized asynchronous data transmission mechanism in data parallel language compilers. This paper proposes an asynchronous data transmission mechanism for data parallel languages. It describes a new linguistic support, called an N-level message queue that designed to overlap communication and computation in a program. Based on an exact data-flow analysis on individual array element accesses, the compiler inserts data transmission requests to message queue into SPMD code. Experiments show that introducing an asynchronous transmission mechanism into data parallel language compilers is effective.