ReDL: A Hybrid Memory System with Scalable Data Management Strategies for DNN Applications

Memory resource is a critical bottleneck for large-scale Deep Neural Network (DNN) applications. Hybrid Memory System (HMS) provides a promising solution to increase memory capacity in an affordable way. However, to release the power of HMS, data migration plays an important role. Deploying DNN on HMS imposes enormous challenges on data migration strategy and inspires us to pursue smart solutions. We propose a runtime system for HMS that automatically optimizes scalable data migrations and exploits domain knowledge on DNNs to manage data between fast and slow memory in HMS. We also introduce a reference distance and location based data management strategy (ReDL), treating short-lived and long-lived data objects with Idle and Dynamic migration methods, respectively. The experimental results demonstrate that with configuring the size of fast memory to be 20% of each workload’s peak memory consumption, our work achieves similar performance to the fast memory-only system.