Optimal hot spot allocation on meshes for large-scale data-parallel algorithms

Hot spots are notorious for degrading the performance of a parallel algorithm. We attempt to minimize the hot-spot access time for a class of problems, namely, Large-Scale Data-Parallel (LSDP) algorithms, on a 2D mesh. An LSDP algorithm has rich data parallelism but without exclusive task and data partitioning. Our approach is to allocate the hot spots at the optimal locations such that the hot-spot access time is minimized. Also, we have designed scheduling algorithms which control hot spot access sequences to achieve the minimal access time. Both uniform and nonuniform hot spots have been considered in this study. We have analytically derived the optimal allocations for wrapped-around and non-wrapped-around square meshes. The theoretical results have been verified by parallelizing the EM algorithm for 3D PET image reconstruction on the Intel iPSC/860