The Fused Multiply-Add and Global Atmospheric Models: A Distributional Investigation into a Surprising Correctness Scenario

In a series of related works developing an ensemble consistency testing approach for multiple popular global climate models (GCMs), one test scenario has repeatedly stood out. Why does the use of the Fused Multiply-Add (FMA) operation result in model configurations getting flagged as failures, while changes to compiler choice, optimization level, processor type and number, etc. are passed as expected? This work explores the impacts of FMA on GCM simulation output from a distributional perspective and provides directions for future work to enable model developers and users to use numerical optimization techniques with confidence.