Quantifying the performance impact of large pages on in-memory big-data workloads

In-memory big-data processing is rapidly emerging as a promising solution for large-scale data analytics with high-performance and/or real-time requirements. In-memory big-data workloads are often hosted on servers that consist of a few multi-core CPUs and large physical memory, exhibiting the non-uniform memory access (NUMA) characteristics. While large pages are commonly known as an effective technique to reduce the performance overheads of virtual memory and widely supported across the modern hardware and system software stacks, relatively little work has been done to investigate their performance impact on in-memory big-data workloads hosted on NUMA systems. To bridge this gap, this work quantifies the performance impact of large pages on in-memory big-data workloads running on a large-scale NUMA system. Our experimental results show that large pages provide no or little performance gains over the 4KB pages when the in-memory big-data workloads process sufficiently large datasets. In addition, our experimental results show that large pages achieve higher performance gains as the dataset size of the in-memory big-data workloads decreases and the NUMA system scale increases. We also discuss the possible performance optimizations for large pages and estimate the potential performance improvements.