Transformer-Based Models for Remaining Useful Life Prediction of Turbojet Engines

Remaining Useful Life (RUL) prediction is an essential task in predictive maintenance. This study evaluates the performance of transformer-based models for predicting the RUL of turbojet engines using the C-MAPSS dataset. The study proposes a sequence length-based dataset labeling method and evaluates the transformer-based model on regression, with and without pretraining. Different transformer-based ensemble models are evaluated. The results show that a shorter sequence length provides better performance in general. Moreover, pretraining the model does not always improve performance. The study shows the potential of transformer-based ensemble models with a careful choice of combined models. While the transformer-based architecture did not outperform state-of-the-art hybrid DL models in RUL prediction, it outperformed individual models. The study concludes that while the transformer model is state-of-the-art in various natural language processing fields, hybrid deep learning models can outperform it in other applications such as RUL prediction.