Incorporating Historic Knowledge into a Communication Library for Self-Optimizing High Performance Computing Applications

Emerging computing systems have a wide variety of hardware and software components influencing the performance of parallel applications, presenting end-users with a (nearly) unique execution environment on each parallel machine. One of the big challenges of High Performance Computing is therefore to develop portable and efficient codes for any execution environment. The Abstract Data and Communication Library (ADCL) is a self-optimizing runtime communication library aiming at providing the highest possible performance for application level communication operations. The library provides for a given communication pattern a large number of implementations and incorporates a runtime selection logic. This selection aims at adaptively choosing the best performing implementation on the current platform and for the given problem. In this paper, we present a recent enhancement to the library which introduces the capability of utilizing information from previous executions in order to minimize the overhead of the runtime selection logic which mainly stems from testing under performing implementations. We introduce the notion of similar problems by using a proximity measure for a given operation. The approach is evaluated for the n-dimensional neighborhood communication for two different network interconnects and for a large range of different problems.