Adventures with clustered filesystems

Current trends in High Performance Computing (HPC) platforms suggest the adoption of the distributed memory paradigm by employing multiple machines for peak floating point performances. This trend must cope with the issue of reading and writing back data on clustered file systems providing shared storage. Clustered file system are usually implemented on a Storage Area Network (SAN) equipped with an optical fiber interconnect with the Computing nodes. Available clustered files systems provide performance benchmarks not always suited to cover all the needs of the users community. Besides providing a basic description of the most widespread clustered filesystems, our main goal is to establish a portable and easy to use benchmark to gauge the clustered file system performances under a typical I/O load for data arranged into a tree composed by thousands of directory i-nodes. A typical example is provided by a graph based statistical database. Results are quite interesting because it is shown that even a 16 nodes symmetric clustered file systems like GFS2 and OCFS2 show operation times much higher than those provided by the most resource consuming Lustre file system. Tables like the ones we show are very useful for providing quantitative insights aimed at driving adoption choices. In our infrastructure, the adoption of an asymmetric file system like Lustre entails a price of 4 computing nodes that are now devoted as centralized file system Metadata servers and other Lustre related services.