The UCLA AGCM in High Performance Computing Environments

General Circulation Models (GCMs) are at the top of the hierarchy of numerical models that are used to study the Earth's climate. To increase the significance of predictions using GCMs requires ensembles of integrations that in turn demand large amounts of computing resources. GCMs codes are particularly difficult to optimize in view of their heterogeneity. In this paper we focus on code optimization for GCMs of the atmosphere (AGCMs), one of the major components of the climate system. In this paper, we present our efforts in optimizing the parallel UCLA AGCM code. The UCLA AGCM is a state-of-the-art finite-difference model of the global atmosphere. Our optimization efforts include the implementation of load balancing schemes, new physical parameterizations of atmospheric processes, code restructuring and use of special mathematical functions. At the beginning of this work, the overall execution time of the code was 459 seconds per simulated day in 256 nodes of a CRAY T3D. At present, the same model configuration requires 51 seconds per simulated day in 256 nodes of a CRAY T3E-900, which is approximately 9 times faster. The peak model performance is about 40 GFLOPs on 512 T3E-900 nodes. We present results in support of our conclusion that major advances in our ability to carry out longer and more detailed climate simulations depend primarily upon development of more powerful supercomputers and that code optimization, for a particular computer architecture, and development of more efficient algorithms can be nearly as important.