Availability Modeling and Assurance of Map-Reduce Computing

This paper proposes a new analytical model to evaluate the availability of map-reduce computing on a Hadoop platform. Map-reduce computing is represented by a queueing model in this work in order to trace flow of tasks (either map or reduce) of their arrivals and exits in the course of computation. The objective of the model is to evaluate the probability for a map-reduce computation to be available at an instant of time, referred to as availability. The set of variables taken into account in this model lists the number of map and reduce tasks, the number of servers (or referred to as nodes in this paper) engaged, along with a few constants such as task arrival/exit rates and node failure/repair rates. The proposed model provides a comprehensive yet fundamental basis to assure and ultimately optimize the design of map-reduce computing in terms of availability with reference to its performance in a simultaneous manner. Parametric simulations have been conducted and demonstrated efficacy pf the proposed model in assessing the availability and the cost for achieving the availability with respect to throughput.