A Hierarchical Approach to Maximizing MapReduce Efficiency

In this paper, we argued that Hadoop has limitations in exploiting data locality and task parallelism for multi-core platforms. We then extended Hadoop with a hierarchical MapReduce scheme. An in-memory cache scheme is also seamlessly integrated to cache data that is likely to be accessed in memory. Evaluation showed that the hierarchical scheme outperforms Hadoop ranging from 1.4x to 3.5x.