Improving Key-Value Cache Performance With Heterogeneous Memory Tiering: A Case Study of Compute-Express-Link-Based Memory Expansion

Compute Express Link (CXL) memory brings extra bandwidth and capacity via Peripheral-Component-Interconnect-Express-based memory expansion beyond double-data-rate-based dynamic random-access memory. This article introduces the CXL 2.0 memory expansion solution, which incorporates two parts: 1) a CXL memory expander prototype and 2) the heterogeneous memory software development kit. We demonstrate the feasibility of our CXL memory solution by implementing it on CacheLib, Meta’s general-purpose key-value caching engine. We highlight how our application design and guidelines for CXL memory enable resolving the shortcomings of conventional memory system architectures. Our proposals enable 1) expanding memory bandwidth and capacity or 2) considerable DRAM savings. Evaluation results show that we can achieve a 25% increase in memory bandwidth, up to 15% throughput gain, and a 9% latency reduction. Furthermore, in hybrid cache using nonvolatile memory (NVM), expanding the RAM cache area with CXL memory, which is relatively cheaper than DRAM, enhances the throughput and hit ratio due to reduced NVM input–output.