Image Distribution Mechanisms in Large Scale Cloud Providers

This paper presents the various mechanisms for virtual machine image distribution within a large batch farm and between sites that offer cloud computing services. The work is presented within the context of the Large Hadron Collider Computing Grid (LCG), it has two main goals. First it aims at presenting the CERN specific mechanisms that have been put in place to test the pre-staging of virtual machine images within a large cloud infrastructure of several hundred physical hosts. Second it introduces the basis of a policy for trusting and distributing virtual machine images between sites of the LCG. Finally experimental results are shown for the distribution of a 10 GB virtual machine image distributed to over 400 physical nodes using a binary tree and a Bit Torrent algorithm. Results show that images can be pre-staged within 30 minutes.