A new Ad-Hoc parallel file system for HPC environments based on the Expand parallel file system

An Ad-Hoc File System dynamically virtualizes storage on compute nodes into a fast storage volume to reduce congestion on parallel file systems used as backends in HPC environments and improve data locality. This paper presents Expand Ad-Hoc, a version of the Expand parallel file system, for use as an Ad-Hoc storage system for HPC environments. Such an update seeks to take better advantage of new storage technologies (on SSDs local to nodes, for example) and to adapt to new demands of current parallel applications (e.g., optimizing data access by analyzing data locality). The paper describes the new system’s features and presents a first evaluation comparing the performance of Expand with another Ad-Hoc File System (GekkoFS), and GPFS. The first results are quite satisfactory and demonstrate the good performance of Expand as an Ad-Hoc storage system.