Spectrum Transformer: An Attention-based Wideband Spectrum Detector

Data-driven machine learning techniques have been advocated for signal detection in complex wireless environments. However, when applied to wideband spectrum sensing scenarios, they face practical challenges including very large data dimensionality, insufficient training data, and implicit inter-band dependencies. Current literature focuses on deep convolutional models, whose inherent model structure is not well suited for representing the diverse spectrum occupancy patterns of practical wideband networks, causing inefficient performance-complexity tradeoff and excessive sensing time. To address these issues, this paper develops a novel Spectrum Transformer with multi-task learning for wideband spectrum sensing at high sample efficiency. Empowered by the multi-head self-attention mechanism, the transformer architecture is designed to effectively learn both the inner-band spectral features and the inter-band spectrum occupancy correlations in the wideband regime. Simulations show that the proposed Spectrum Transformer outperforms the existing methods based on convolutional neural networks especially in the small-data case, by achieving higher sensing accuracy with an 89% reduction in model complexity.