A Holistic Heterogeneity-Aware Data Placement Scheme for Hybrid Parallel I/O Systems

We present
H2DP
, a holistic heterogeneity-aware data placement scheme for hybrid parallel I/O systems, which consist of HDD servers and SSD servers. Most of the existing approaches focus on server performance or application I/O pattern heterogeneity in data placement.
H2DP
considers three axes of heterogeneity: server performance, server space, and application I/O pattern. More specifically,
H2DP
determines the optimized stripe sizes on servers based on server performance, keeps only critical data on all hybrid servers and the rest data on HDD servers, and dynamically migrates data among different types of servers at run-time. This holistic heterogeneity-awareness enables
H2DP
to achieve high performance by alleviating server load imbalance, efficiently utilizing SSD space, and accommodating application pattern variation. We have implemented a prototype of
H2DP
under MPICH2 atop OrangeFS. Extensive experimental results demonstrate that
H2DP
significantly improve I/O system performance compared to existing data placement schemes.