Hardware acceleration of Hadoop MapReduce

MapReduce is widely used for BigData processing. It was originally designed to overcome the I/O bottleneck of commodity servers. However, several high speed storage and network devices have recently emerged, and speeds continue to increase. Employing such brand new devices will solve the I/O bottleneck, making the CPU the next serious bottleneck in the MapReduce framework. In this paper, we conduct a performance study of Hadoop MapReduce by using a server cluster built on state-of-the-art devices. We show the CPU is the bottleneck in such an environment. To overcome the CPU bottleneck, we propose hardware acceleration for MapReduce. We implement a prototype using a many core processor board developed by Tilera and show the feasibility of our proposal.