Bulk I/O Storage Management for Big Data Applications

We propose and design a new Block I/O schedulingscheme called Bulk I/O Dispatch (BID) suited for disk intensiveMapReduce applications. Large data access by such applicationsresult in a large number of block I/O requests which have thepotential to be sequentialized. However, due to contention byother applications and the way current I/O schedulers operate, the opportunities of a large sequential I/Os are missed. SequentialI/Os, which are faster than random I/Os, can lead to saving theCPU wait times and thus better application performance. The proposed scheduler is designed to work with all blockdevices which have superior sequential performance than random. Through simulation based experiments with MapReducebenchmarks we show that the proposed block I/O schedulerresults in about 27% to 52% lesser time for I/O than the currentlyavailable schedulers.