DeepFetch: A Node-Aware Greedy Fetch System for Distributed Cache of Deep Learning Applications

Data I/O poses a significant bottleneck for distributed deep learning applications. Utilizing computing-node attached storage as a cache has become a prevalent solution to this problem. Given the large size of training datasets and the limited capacity of a single-node local storage, training samples are loaded in shards across computing nodes, requiring a deep learning job on one node to access samples from neighboring nodes. Compared to local node access, these neighboring accesses are highly inefficient for small samples. To mitigate this issue, we propose a node-aware prefetch algorithm that greedily fetches samples from neighboring nodes. Evaluation results show that our approach improves neighboring access performance by 60x for small samples of 64B size. For large samples of 1MB size, our approach still exhibits a 26% improvement.