High-efficiency Lattice QCD computations on the Fermi architecture

Lattice Quantum Chromodynamics (LQCD) is a computationally challenging problem that solves the discretized Dirac equation in the presence of an SU(3) gauge field. The most time-consuming computational routine is the application of the discretized Dirac operator (a sparse matrix) to a vector. GPUs have proven to be a popular platform on which to deploy such computations; however, because of the relatively low arithmetic intensity (flop-to-byte ratio) of this “Dslash” operation, it is hard to realize a significant fraction of peak performance. Like many other stencil-based computations, the Dslash operation exhibits spatial locality, which can be used to minimize memory traffic and hence increase achievable performance. In this work we present a cache-blocking strategy which is suited to the relatively limited amount of cache on current-generation GPUs. We present results on NVIDIA's Fermi architecture for the Wilson discretization, demonstrating performance in excess of 300 Gflops in single precision, which corresponds to 79% of the peak achievable assuming perfect reuse (i.e., an infinite cache). Finally, we speculate on how one might improve on this performance through finer-grained parallelization and temporal cache blocking.