Using Locality-Enhanced Distributed Memory Cache to Accelerate Applications on High Performance Computers

Nowadays high performance computers (HPC) are used to solve increasingly complex problems and process larger amounts of data. The growing computational requirements of applications can be met by utilizing more compute nodes. However, the average I/O performance a compute node can utilize is reduced with increased number of nodes. The performance gap between computation and I/O has long been a primary issue impacting application performance. Distributed memory cache has been proposed to narrow the performance gap by caching data in the memory of multiple compute nodes. However, former approaches didn't fully optimize the performance of accessing locally cached data. We design and implement a locality-enhanced distributed memory cache (LeCache) to address such problem. LeCache separates the location of metadata and data, with which it enables data to be preferentially cached in local memory. The proposed metadata caching strategy further minimizes the overhead of querying metadata remotely. We conduct extensive evaluation with IOR and BTIO in Tianhe-1A. The results show that LeCache has significant performance advantage under various kinds of workloads.