A Scalable Wide-Area Grid Resource Management Framework

Grid computing systems federate resources belonging to several organizations to support applications with large computation and storage needs. Effective resource management is crucial for realizing the promise of the grid. However, current grid resource management frameworks have a number of limitations including poor scalability, and inadequate support for quality of service (QoS). This paper describes a novel and scalable grid resource management framework that can address these limitations. The framework relies on a hierarchical organization of resources and resource managers (RM) within an organization. Resources are assigned to jobs through decentralized inter and intra organizational collaborations between RMs. The framework employs a hierarchical information aggregation scheme that permits scalable grid resource management. Such a capability allows more intelligent placement of workloads across the grid than is feasible with traditional resource managers. For example, loads can be balanced across the grid clusters to avoid over utilization of resources resulting in better QoS for jobs. Hierarchical segmentation of grid resources allows the framework to handle dynamic situations (e.g., failure recovery, and nodes joining the Grid). The improved scalability of the framework, however, comes at the price of incurring additional complexity and overhead. Sophisticated protocols need to be designed to build and provide the functionality of the hierarchy. Considering the benefits and limitations of our approach, we believe the hierarchical framework is best suited for managing planetary scale grid systems supporting "embarrassingly" parallel jobs that require computational resources beyond the borders of an organization