RTLB_Sched: Real Time Load Balancing Scheduler for CPU-GPU Heterogeneous Systems

Heterogeneous computing systems consisting of Central Processing Units CPUs and Graphical Processing Units GPUs are found everywhere, from mobile phones, laptops to cloud clusters, due to their low cost/performance ratio and their impressive peak computational performance. One of the main and most complex problems in these systems is the load imbalance. This imbalance is caused by the large computing power difference between its computing nodes. We present RTLB_Sched, a Real Time Load Balancing Scheduler that minimizes the load difference between the CPU and GPU nodes using machine learning. We define a new metric called load difference using execution time prediction. We describe the method and provide mathematical formulas to calculate this metric. Experiments of scheduling multiple OpenCL applications on a heterogeneous system show that RTLB _Sched outperforms the Device Suitability and Round Robin approaches in load balancing.