Scalable Storage I/O Software for Blue Gene Architectures

This chapter presents a scalable storage I/O software solution for the hierarchical architecture of Blue Gene supercomputers based on collective I/O optimizations, file views, and pipelined data staging. At the compute node level, application processes can declare views on the files as hints of potential future accesses, employ collective I/O operations for reducing the number of file system requests, and use asynchronous I/O for overlapping computations and storage I/O. The chapter also provides an overview of the Blue Gene architecture. It concludes that, in the future, it plans to design novel adaptive data staging policies that address changes in network congestion, I/O node load, and file system response time. It also plans to study how the solution can be adapted to novel architectural designs including nonvolatile RAM (NVRAM) and to explore the role of NVRAM in reducing the impact of I/O bursts and increasing the scalability of the I/O stack.