Efficient task scheduling for Many Task Computing with resource attribute selection

Many Task Computing (MTC) is a new class of computing paradigm in which the aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. With the advent of Cloud computing and big data era, scheduling and executing large-scale computing tasks efficiently and allocating resources to tasks reasonably are becoming a quite challenging problem. To improve both task execution and resource utilization efficiency, we present a task scheduling algorithm with resource attribute selection, which can select the optimal node to execute a task according to its resource requirements and the fitness between the resource node and the task. Experiment results show that there is significant improvement in execution throughput and resource utilization compared with the other three algorithms and four scheduling frameworks. In the scheduling algorithm comparison, the throughput is 77% higher than Min-Min algorithm and the resource utilization can reach 91%. In the scheduling framework comparison, the throughput (with work-stealing) is at least 30% higher than the other frameworks and the resource utilization reaches 94%. The scheduling algorithm can make a good model for practical MTC applications.