A scalable performance prediction heuristic for implementation planning on heterogeneous systems

Despite speedups of 10x to 1000x, effective usage of multi-core and heterogeneous systems has largely been limited to experts due to increased application design complexity resulting from the requirement for significantly different algorithms for different device types and amounts. Compiler and high-level synthesis research has attempted to address this problem but is fundamentally limited to the algorithm specified by the high-level code. Thus, future compilers will need to choose from numerous implementations/algorithms for a given function when optimizing for a multi-core heterogeneous system. This emerging problem, which we refer to as the implementation planning problem, requires compilers and similar tools to rapidly determine performance of a particular implementation on different devices for all possible input parameters. To help solve the implementation planning problem, we introduce a heuristic that repeatedly selects statistically significant input values, measures actual execution time, and then statistically analyzes the results to predict the execution time for all inputs within requested accuracy and confidence levels. We evaluated the heuristic using twelve examples on three different platforms with up to 16 microprocessor cores and a field-programmable gate array, achieving an average prediction error of 6.2% and a root-mean-squared error of 7.4%, which required an average of only 463 samples and 51 seconds to complete.