Transformer Masked Autoencoders for RF Device Fingerprinting

Machine learning methods for radio frequency (RF) device fingerprinting typically rely on convolutional neural net (CNN) models. Transformer-based models have outperformed CNNs for modulation classification tasks, but there are few implementations for device fingerprinting. We train a transformer-based model for device fingerprinting with the largest device count to date and explore several variations of the architecture. Out of these variations, splitting the signal into 64 element patches for embedding provided a good balance between operation count and model accuracy, and using dense layers for pooling provided the highest accuracy. Additionally, we demonstrate that pre-training an RF transformer as a masked autoencoder improves classification accuracy, as has been observed for CNN fingerprinting models and vision transformers.