A Configuration Crawler for Virtual Appliances in Compute Clouds

Compute clouds are pools of virtual machines that are shared in a multi-tenant environment by multiple users. The virtual machine images are stored in one or more repositories and are pre-configured with an operating system. Users of the compute cloud can upload their own images or install and configure additional software on top of existing basic virtual machines. Today, the Amazon Elastic Compute Cloud (EC2) counts more than 35,000 publicly available virtual machine images. We observe, however, that the meta-data that describes the virtual machine images is of poor quality and does not cover vital information such as operating system configurations or software package installations. The sprawl of poorly documented virtual machine images poses a hurdle to sharing and re-use among members of the compute cloud community. We present a method that allows collecting software-related meta-data in compute clouds through appliance introspection. Moreover, we show how applications in the domains of selection and configuration management benefit from rich meta-data and interact with the method. The method has been implemented as an automated tool, the crawler, that collects configuration data of virtual machine images in public compute clouds and evaluated our approach by crawling Amazon EC2.