Hound: A Parallel Image Distribution System for Cluster Based on Docker

Current applications, consisting of multiple replicas, are packaged into lightweight containers with their execution dependencies. Considering the dominant impact of distribution efficiency of gigantic images on container startup (e.g., distributed deep learning application), the image “warm-up” technique which prefetches images of these replicas to destination nodes in the cluster is proposed. However, the current image “warm-up” technique solely focuses on identical image distribution, which fails to take effect when distributing different images to destination nodes. To address this problem, this paper proposes Hound, a simple but efficient cluster image distribution system based on Docker. To support diverse image distribution requests of cluster nodes, Hound additionally adopts node-level parallelism (i.e., downloading images to destination nodes in parallel) to further improve the efficiency of image distribution. The experimental results demonstrate Hound outperforms Docker, kubernetes container runtime interface (CRI-O), and Docker-compose in terms of image distribution performance when cluster nodes request different images. Moreover, the high scalability of Hound is evaluated in the scenario of ten nodes.