CacheCraft: Enhancing GPU Performance under Memory Protection through Reconstructed Caching

Contemporary GPUs use Error Correcting Codes (ECC) to protect against memory errors. GPUs with Graphics DDR (GDDR) utilize in-band ECC (a.k.a. inline ECC), which sequentially accesses data and redundancy to enable ECC functionality using non-ECC memory chips. However, the additional access reduces data throughput and can incur significant performance penalties for bandwidth-intensive applications. This paper introduces CacheCraft, a novel GPU micro-architecture engineered to address the inefficiencies of current in-band ECC protection. It reconfigures the traditional 128B cache line from four 32B sectors into four 30B sectors and one 8B sector. This adjustment creates a 2B space in each 32B memory chunk, designated for storing the redundancy of the sector data, thereby enabling a single memory access to deliver reliable data. Our evaluation shows that this single-access in-band ECC can significantly mitigate the bandwidth penalty of memory protection. While traditional in-band ECC increases memory access by 41.9% (peaking at 96.9%), CacheCraft reduces this extra bandwidth requirement to 21.9 % (peaking at 28.2 %). This significant reduction (47.8 % on average and up to 89.4 %) can substantially enhance the performance of memory-intensive applications by as much as 23.5 %.