Satellite Anomaly Detection based on Improved Transformer Method

In this paper, we present an Improved Transformer model and a self-supervised framework for detecting anomalies in satellite telemetry data. Previous approaches based on RNN and LSTM variants have shown some progress in anomaly detection, but still require improvement in detection performance and context parallelism. To address these challenges, we propose an innovative Improved Transformer model that incorporates correlation modeling, long-range modeling, and parallel reasoning capabilities. This model features a token split structure, position embedding, and mask unit, which enhance its overall performance. Our self-supervised framework consists of two stages: pre-training and fine-tuning. We conduct extensive experiments on NASA’s public dataset and demonstrate the effectiveness of our proposed framework. Using our self-supervised approach, we achieved a significant 95.7% (3-class) F1-score on the validation set, outperforming supervised counterparts that utilized the same training data.