By Topic

MiF: Mitigating the Intra-file Fragmentation in Parallel File System

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

5 Author(s)
Letian Yi ; Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China ; Jiwu Shu ; Youyou Lu ; Wei Wang
more authors

Parallel file systems have been broadly deployed in large scale data centers, supporting a wide range of applications across a variety of industries. Unfortunately, most parallel file systems suffer from the intra-file fragmentation which is the disk performance killer. This paper presents the design and implementation of MiF, which introduces two techniques: on-demand preallocation and embedded directory, to Mitigate the intra-file Fragmentation of data placement, improving the disk performance in parallel file system. The key insight of on-demand preallocation is that the preallocation of a file should be aware of concurrent process streams and recognizes the write characteristic. The background rationale of embedded directory is that, since modern parallel file systems aggregate many normal operation pairs, exploring the disk bandwidth for metadata access requires all metadata of sub-flies in the same directory be placed adjacently. Measurements of our MiF implementation in a block-based parallel file system demonstrate that it can significantly improve I/O performance of parallel programs.

Published in:

Parallel Processing (ICPP), 2011 International Conference on

Date of Conference:

13-16 Sept. 2011