Characterizing data organization effects on heterogeneous memory architectures

Layout and placement of shared data structures is critical to achieving scalable performance on heterogeneous memory architectures. While recent research has established the importance of data organization and developed mechanisms for data layout conversion, a general strategy for when to make layout changes and where to map data segments in a heterogeneous environment, has not yet emerged. In this paper, we present a cross-platform study that characterizes the performance impact of data organization on candidate HPC node architectures with heterogeneous, multi-level memory systems. We systematically explore a multidimensional space of alternate code variants, identify program attributes that have the most impact on data organization decisions and establish a set of performance imperatives to guide data layout and placement across different architectures. The study shows that the conventional approach of using a structure-of-arrays for device-mapped data structures is not always profitable and that in addition to memory divergence, data layout choices are impacted by a variety of factors including arithmetic intensity and register pressure. We use the results to develop a new data layout strategy that addresses the limitations of current approaches and yields up to an order-of-magnitude speedup on some architectures.