EFI: Cache Replacement Policy Using Eviction Frequency Integration

Modern processors have last-level caches (LLCs) that are unsuccessful in capturing large amounts of data from memory-intensive workloads. The replacement policy of the LLC is a significant factor in the performance potential of the processor since its selection determines the frequency of off-chip misses for the processor. Existing processors utilize variations of the least-recently-used (LRU) and not-recently-used (NRU) replacement policies for victim selection. However, due to the nature of LRU and NRU, they are only efficient for workloads with high temporal locality and working sets smaller than the available cache. When we look at Belady's MIN algorithm, the optimal replacement policy, we observe that LRU is not sufficient in mirroring this efficiency for thrash and scan-resistant workloads. We find that in prior work, there exists an inverse relationship between the expected number of hits of a cache block and its reuse distance. Building off of this observation, we propose that integrating this characteristic with the eviction frequency of a line from the cache can improve the efficiency of LLCs and as a result give us an effective replacement policy. We suggest an addition to the hit-count-based selection policy on top of low-cost replacement policies to offer performance improvement to the replacement policy of the last-level cache. The experiment results show that 2.3 % performance improvement over the Expected Hit Count Replacement Policy, which offers 12.9% cache performance improvement over Baseline LRU.