SKERD: Reuse distance analysis for simultaneous multiple GPU kernel executions

Modern GPUs employ simultaneous kernel executions (SKE), an equivalent to multitasking in CPUs, to maximize the hardware utilization and enhance the resulted performance. SKE paradigm is not yet fully explored by the research community. In this study, a reuse-distance (RD) based analysis approach, called SKERD, is proposed to analyze the effect of SKE scenarios on the kernel data reuse and GPU cache memories performance. Only two simultaneous kernels were considered in this work. Moreover, Three types of coarse-grained SM (streaming multiprocessor) partitioning schemes were investigated including an even SM to kernel partitioning and two SM partitioning schemes that assign the SMs to the kernels based on the kernel workloads. The simulation results show that none of the mentioned partitioning schemes always functions better than the others. Further, for some memory intensive kernels, SKE resulted in cache contentions and hit ratio degradation. Consequently, the effects of SKE on cache memories should be carefully considered.