Data Layout Organization effect on Communication in Parallel Programs

In the new era of computing, data movement is considered the dominant factor influencing both performance and power/energy consumption. This movement could be between different nodes or between different levels of memory in the same node. It could decrease requests to fetch data from memory to higher speed cache memory, resulting in a decrease in cache miss ratio. Additionally, it may decrease the energy/power consumption. This assumption applies to a wide spectrum of architectures such as single core, multicores, many-cores, multi-nodes, warehouse scale computing, and Domain Specific Architectures (DSA). The revolutionary approach of hardware manufacturing suggests that the gap between computing, and data movements, both in memory subsystems and interconnection networks will continue to widen. Therefore, minimizing communications is an essential goal to achieve better utilization of modern architectures. In this paper, we study the impact of the data layout re-organization to minimize communications in parallel programs. The proposed technique is based on the ordering and partitioning of the Value Communications Graph (VCG) as a representative of the communications in the program. The evaluation work is carried out using a stack distance analysis of the program. The ordering and partitioning of the VCG is conducted using METIS, a graph ordering and partitioning tool.