Lightweight Transformer Model for Winter Wheat Yield Prediction Based on Multi-source Data

To enable multidimensional agricultural monitoring, this paper presents a lightweight transformer model for winter wheat yield estimation, which is based on the cell structure of the original transformer network, replacing the decoding layer with a linear layer, and trained using multiple sources of data such as crop growth variables and environmental variables from 2002 to 2020. Our approach is compared with other methods, including long and short term memory (A-LSTM) and gated recursive unit (GRU) models, introduced by a self-focused mechanism. The results show that the proposed lightweight transformer model can achieve considerable accuracy ($R^{2}$=0.9483, RMSE =130.5820, MAE = 107.5848 and MAPE =0.1783). Compared to the mainstream temporal prediction methods (traditional recurrent neural networks), the proposed model improves the prediction accuracy by 18.98% and 10.33% respectively. Indicators such as original band information and vegetation had a greater impact on winter wheat yield prediction than other environmental factors. This study provides a feasible avenue for large-scale yield prediction.