Wait-Free Programming for General Purpose Computations on Graphics Processors

The fact that graphics processors (GPUs) are today’s most powerful computational hardware for the dollar has motivated researchers to utilize the ubiquitous and powerful GPUs for general-purpose computing. However, unlike CPUs, GPUs are optimized for processing 3D graphics (e.g., graphics rendering), a kind of data-parallel applications, and consequently, several GPUs do not support strong synchronization primitives to coordinate their cores. This prevents the GPUs from being deployed more widely for general-purpose computing. This paper aims at bridging the gap between the lack of strong synchronization primitives in the GPUs and the need for strong synchronization mechanisms in parallel applications. Based on the intrinsic features of typical GPU architectures, we construct strong synchronization objects such as wait-free and
$t$
-resilient
read-modify-write
objects for a general model of GPU architectures without hardware synchronization primitives such as
test-and-set
and
compare-and-swap
. Accesses to the wait-free objects have time complexity
$O(N)$
, where
$N$
is the number of processes. The wait-free objects have the optimal space complexity
$O(N^2)$
. Our result demonstrates that it is possible to construct wait-free synchronization mechanisms for GPUs without strong synchronization primitives in hardware and that wait-free programming is possible for such GPUs.