Training and Optimization of Spiking Transformer Models Under ReRAM Non-Ideal Effects

Memristor-based spiking neural networks (SNNs) offer energy efficiency and area compactness, yet most existing studies focus on CNN-based models with idealized hardware characteristics. This paper introduces a five-stage training framework for transformer-based models on ReRAM architectures, addressing critical non-ideal effects, including nonlinearity and variability. The proposed approach incorporates quantization, resistance modeling, and post-training variability tuning to enhance model robustness and accuracy. Experimental results demonstrate accuracy improvements of $1. 4 5 \%, 9. 1 8 \%$, and 19.22 % on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets, respectively, under non-ideal ReRAM effects. These results validate the effectiveness of the proposed method in advancing transformer-based SNNs for efficient neuromorphic computing.