The Scalability of Embedded Structured Grids and Unstructured Grids in Large Scale Ice Sheet Modeling on Distributed Memory Parallel Computers

Currently, there are two approaches to implementing large-scale ice sheet simulations. One approach, which is based on the use of an evenly-spaced structured grid, is to simulate the entire ice sheet at the highest resolution required. The other approach is to model the ice sheet at different levels of resolution using unstructured meshes. Structured grids lend themselves quite well to partitioning on multi-core, distributed memory parallel computers, leading to good computational efficiency, load-balancing and reduced communication costs. The disadvantage is the significant computational resources required to model a system of this size at a high enough resolution to correctly model regions of the ice sheet undergoing rapid change. Unstructured meshes can reduce such costs by allocating more computational resources to areas of the ice sheet whose dynamics are evolving quickly and fewer resources to areas evolving more slowly. However, unstructured meshes do not easily lend themselves to effective decomposition on distributed memory parallel computers, and can experience poor cache utilization, significant load imbalance, high communication costs, and increased synchronization times. In this paper, we describe our alternative approach based on embedded simulation, which provides a degree of multi-resolution capabilities to structured grids while maintaining many of the computational efficiencies that come with the structured approach. We provide experimental results focusing on the scalability characteristics and cache efficiency of each approach as a function of increasing problem size and core counts. We show that the embedded approach scales very well across all problem sizes and core counts, and that the unstructured approach scales well at smaller problem sizes and core counts but degrades significantly as the problem size becomes large.