A performance-satisfied and affection-aware MapReduce allocation scheme for intelligent information applications

Intelligent information applications (e.g. healthcare, business data mining, etc.) usually involve the processing of a huge amount of data. MapReduce can speed up the execution of the application (job) with big data by dividing the job into a number of concurrently running map and reduce tasks in cloud computing systems. With many MapReduce jobs in the systems, it is required to efficiently allocate computing resources for the tasks of such jobs. If not, the performance requirements (e.g. execution deadlines) of some jobs cannot be met. Several deadline-constrained MapReduce schedulers have been proposed, but they do not consider the following factors: 1) affection on existing tasks, and 2) the tradeoff between the number of tasks allocated and the total allocation cost. To take into consideration the above three factors in the MapReduce scheduling, we transform the task allocation problem into a well-known network graph problem: minimum cost flow (MCF) to solve our concerned task allocation problem. Finally, the performance analysis is done to demonstrate the effectiveness of our proposed MapReduce scheduler.