A Characterization of the Effects of Software Instruction Prefetching on an Aggressive Front-end

Growing application sizes continue to strain the memory system. As more complex applications are developed, and the instruction memory footprint increases, the cache hierarchy cannot contain relevant instructions causing the front end to become idle as it awaits fetched instructions. Hardware instruction prefetchers can alleviate this problem by learning instruction stream behavior and prefetching instructions into the cache before use. Fetch Directed Prefetching (FDP) is a ubiquitous form of hardware instruction prefetching that uses branch predictor to predict future instruction cache references. Modern processors generally implement aggressive, deep FDP to decouple the front-end from the rest of the machine. Instruction accesses, however, provide a small amount of information over long periods due to instruction stream variability and lack of information regarding the context of instruction accesses. Capturing the instruction stream’s context requires significant storage overhead to correlate instruction accesses and ensure timely accesses. Software prefetching techniques overcome this problem by profiling and statically analyzing an application’s behavior. Prior work demonstrates software instruction prefetching’s potential performance benefit but does not evaluate performance in the context of aggressive, decoupled front-ends. While software prefetching provides ~ 20% improvement in conservative front-ends, we find that it does not yield performance benefit when modeling a baseline with an aggressive FDP, in some cases hurting performance. Our analysis finds that using software instruction prefetching negatively impacts an aggressive front-end’s behavior. We investigate this finding and characterize the different states a front-end can be in and how introducing instructions into an application can change the front-end’s behavior resulting in destructive interference with the software prefetcher.