An Efficient Transformer Model Enhanced by S-Transform and Transfer Learning for Predicting Gas Distribution in Deeply Buried Reservoirs

In predicting gas-bearing potential in deeply buried reservoirs, traditional time-frequency analysis struggles with weak seismic responses. Inspired by transformers in the text translation, this study develops an efficient transformer model that adaptively captures gas-sensitive frequency features through multihead self-attention (MSA) mechanisms. The framework operates on 1-D instantaneous amplitude spectra with binary gas-bearing/gas-free labels, generating probabilistic gas presence predictions. Methodologically, the approach combines S-transform (ST) for high-resolution time-frequency decomposition with transfer learning for overfitting mitigation. Implementation proceeds through three stages: First, ST decomposes seismic time data to analyze frequency energy distributions, extracting main frequency intervals for dataset construction. Second, an efficient transformer model is built, where the seismic amplitude spectrum is segmented using a Segment Embedding layer and transformed into high-dimensional vectors. These vectors, combined with classification tokens and learnable positional encoding, are processed by efficient MSA (EMSA) to enhance interhead information flow. The classification output is derived from the final hidden state of the classification token via linear classification and softmax layers. Third, transfer learning involves synthetic noisy data pretraining, interpolation-based model adaptation, and low-learning-rate fine-tuning. Experimentally, the pretrained model demonstrated robust noise tolerance, achieving 95% accuracy on the test dataset containing noisy ( $- 15\sim 5$ dB SNR) and clean samples. After fine-tuning, the final model achieved a testing accuracy exceeding 85% of seismic data from the Sichuan Basin in China. The predicted gas-bearing reservoir distribution closely matched the well-logging data, demonstrating robust generalization and transferability across diverse geological domains.