The Effects of Wide Vector Operations on Processor Caches

In this paper we investigate the effects of wide vector instructions on modern processor caches. On the one hand, contemporary processors have large, highly associative caches, which greatly benefit applications that can exploit spatial or temporal data locality. On the other hand, vector instructions operate on wide lists of operands, and moving this data through the cache hierarchy can fill it up quickly. We use a selection of mini-apps representative of a range of scientific application classes to investigate the behaviour of caches in two state-of-the-art Arm-based processors, the Marvell ThunderX2 and the Fujitsu A64FX. We compile the application to target the Arm Scalable Vector Extension (SVE) and we model the caches of these two processors using a newly developed cache simulator. We then vary a number of cache parameters and show how these choice influence application behaviour at a range of SVE widths between 128 and 2048 bits. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space. We use a selection of mini-apps representative of a range of scientific application classes to investigate the behaviour of caches in two state-of-the-art Arm-based processors, the Marvell ThunderX2 and the Fujitsu A64FX. We compile the application to target the Arm Scalable Vector Extension (SVE) and we model the caches of these two processors using a newly developed cache simulator. We then vary a number of cache parameters and show how these choice influence application behaviour at a range of SVE widths between 128 and 2048 bits. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space. We observed a correlation between higher cache associativity and lower miss rate. For the first cache level, at higher line sizes an increase in associativity was necessary to decrease miss rate compared to a cache with the same total size but smaller line size; for the second level, higher associativity did not always result in better performance with long cache lines. As the SVE width was scaled, data was evicted from cache quicker, an effect which was more noticeable at smaller line sizes. Larger cache lines also allowed non-contiguous requests to be fulfilled with fewer loads, because each cache lines covers more memory space.