Remote application scheduling on metacomputing systems

Efficient and robust metacomputing requires the decomposition of complex jobs into tasks that must be scheduled on distributed processing nodes. There are various ways of creating a schedule and implementing it efficiently, depending upon global system state knowledge. Many computations may be structured as process networks, where data is either pushed from the source node to the target node where it will be used, or is pulled from source to target at the instigation of the target. We have developed a metacomputing infrastructure to investigate this idea, which employs the concept of a rich data pointer, the DISCWorld Remote Access Mechanism (DRAM), which can point to either data or services, and can be traded in a client/multiple-server model. We present an extension of the DRAM concept and implementation to represent and describe data that has not yet been created, the "DRAM Future" (DRAMF). We show how the use of the DRAMF facilitates efficient metacomputing scheduling and runtime optimisation on high performance distributed systems. We present a recursive algorithm for determining the optimal placement of a job's components in the presence of partial system state information. This algorithm uses only a selected subset of all available processing nodes, and we implement it using DRAMFs. There are many research issues to consider when designing a robust and general algorithm for scheduling and process placement on distributed systems. We address some of these issues in our Distributed Information Systems Control World project[6] as do other research projects[2].