MDLoader: A Hybrid Model-driven Data Loader for Distributed Deep Neural Networks Training

In this work, we propose MD Loader, a hybrid in-memory data loader for distributed deep neural networks. MDLoader introduces a model-driven performance estimator to automatically switch between one-sided and collective communication at runtime.