Experiences with Hierarchical Storage Management Support in Blue Whale File System

In order to meet the challenges of significant storage and application growth, as well as shortened backup windows and limited IT resources, more and more organizations embrace Hierarchical Storage Management (HSM). Parts of SAN file systems provide the ability to support HSM, but they didn't provide more details in design considerations and how to implement. In this paper, we present design considerations in Blue Whale File System (BWFS) which is a high performance SAN file system and implement the system. Event, migration and attribute constitute the three aspects of data management mechanism in BWFS. As for READ/WRITE/TRUNCATE event, the problem is that a client directly accesses to storage devices containing file data, rather than through a metadata server. We monitor LayoutGet operation instead of real I/O path. BWFS is a multi-volume file system which can support single name space across multiple data volumes. When different storage devices are managed by BWFS, migration is between volumes in BWFS. We also implement migration between BWFS and external repository. We believe the solutions to the problem we encountered are useful to file system developers.