Adaptive Distributed Job Scheduler for Java Compute Servers

Federation of shared memory space type distributed system with the basic command pattern has yielded a very powerful distributed systems architecture, which is known as Java compute servers. Java compute servers leverage the power of shared memory space based distributed systems in addition to the ability to perform any activity\computation by any processing node within the system. Within this distributed environment there are set of processors known as workers. The task of the workers is to perform the task available in the shared memory space. These workers randomly read these entries in the shared memory space and perform the computation and write back the results in the shared space. These results will be picked up by the master process and if required aggregate the results into one result and deliver to the source of the task. This random selection of tasks from the shared memory space by the worker processes, can lead to low performance in the distributed system, if computational intensive tasks are taken up by slow workers on slower nodes. The solution to the above problem is to implement a job scheduling framework for Java compute servers. This paper proposes an adaptive job scheduling framework which takes into account the processing capabilities of nodes and the processing requirements of tasks when scheduling tasks. To evaluate the framework in a practical environment, the proposed scheduling framework was implemented for an image processing application.