NPACI: rocks: tools and techniques for easily deploying manageable Linux clusters

High-performance computing clusters (commodity hardware with low-latency, high-bandwidth interconnects) based on Linux, are rapidly becoming the dominant computing platform for a wide range of scientific disciplines. Yet, straightforward software installation, maintenance, and health monitoring for large-scale clusters has been a consistent and nagging problem for non-cluster experts. The NPACI Rocks toolkit takes a fresh perspective on management and installation of clusters to dramatically simplify software version tracking, and cluster integration. The toolkit incorporates the latest Red Hat distribution (including security patches) with additional cluster-specific software. Using the identical software tools used to create the base distribution, users can customize and localize Rocks for their site. Strong adherence to widely-used (de facto) tools allows Rocks to move with the rapid pace of Linux development. Version 2.1 of the toolkit is available for download and installation. To date, 10 clusters spread among 5 institutions have been built using this toolkit.