Peripheral memory: a technique for fighting memory bandwidth bottleneck

Memory bottleneck has always been a major cause for limiting the performance of computer systems. While in the past latency was the major concern, today, lack of bandwidth becomes a limiting factor as well, as a result of exploiting more parallelism with the growing number of cores per die, which intensifies the pressure on the memory bus. In such an environment, any additional traffic to memory, such as the I/O traffic may lead to degradation of the overall performance of the system. This work introduces the concept of Peripheral Memory, a software controlled memory that resides in the I/O domain and can be used for offloading I/O traffic from CPU memory. The Peripheral Memory handles ‘I/O exclusive data’, data originated and terminated at I/O domain, and which does not need any processing by the CPU. The paper analyses the impact of I/O traffic on the overall performance of the current systems and demonstrates that in numerous applications, I/O exclusive data occupies major part of memory bandwidth, as a result, degrading CPU processing performance and increasing power. The paper considers four different implementations of the Peripheral Memory: pageable, pinned, non-coherent split-traffic and copy-on-access. Our full-system simulator indicates that non-coherent split traffic configuration is the most efficient implementation, which can provide up to four times speedup in the I/O processing rate for typical I/O intensive applications. In addition, based on Power model and measurements tools, the paper demonstrates that the Peripheral Memory in a server system can lead to reduction of tens of Watts in the overall system power consumption or 10-20 percent of the system power budget.