HEET: A Performance Measure to Quantify Heterogeneity in Distributed Computing Systems

Despite extensive research on system heterogeneity, there remains a gap in measuring its impact on system performance. Previous studies have primarily focused on the binary definition of heterogeneity based on architectural diversity, without exploring dimensions of system heterogeneity and their impacts on the system performance. To bridge this gap, in this study, we propose a heterogeneity measure that, for a given heterogeneous system, offers a representation of its performance. This heterogeneity measure is instrumental for solution architects in proactively defining their systems to be sufficiently heterogeneous to meet their desired performance objectives. Accordingly, we develop a mathematical model to characterize a heterogeneous system in terms of its task and machine heterogeneity dimensions and then condense it to a single value, called Homogeneous Equivalent Execution Time (HEET), which represents the execution time behavior of the entire system. We used AWS EC2 instances to implement a machine learning inference system. Using HEET scores in various heterogeneous system configurations demonstrated that HEET can accurately characterize the performance behavior of these systems. In particular, the results show that HEET can help predict the true makespan of heterogeneous systems with an average precision of 84%.