Adversarial Domain Generalization Network for New Enabled Hard Disk Drive Failure Prediction

Disk failures in large data centers are a common occurrence and can result in significant data loss. As new disks are gradually incorporated into the storage system over time to expand its capacity or replace faulty disks, a small number of new disk models may appear. This poses a challenge for disk failure prediction in data centers, as traditional methods struggle to provide satisfactory predictive performance in evolving storage systems due to the lack of sufficient training data for these new disk types. To address this issue, a novel domain generalization network, named the adversarial long short-term memory (LSTM)-autoencoder (ALAE), is proposed for predicting failures in newly enabled hard disk drives (HDDs). Our proposed method utilizes the LSTM-autoencoder to effectively capture temporal dependencies in sequential data, and employs the maximum mean discrepancy (MMD) metric to align the distribution of features across different disk types and improve the generalization ability of the model. Furthermore, the adversarial learning between a prior distribution and the latent codes generated by the encoder is introduced to prevent overfitting of the learned features. The effectiveness of our proposed method is validated on the Backblaze dataset for HDD failure prediction, with experimental results demonstrating its superior performance compared to other state-of-the-art methods.