Computationally Efficient Single Layer Transformer Convolutional Encoder for Accurate Price Prediction of Agriculture Commodities

Predicting the accurate future price of the agricultural crops is important to avoid overproduction or shortages in the food supply chain. To obtain accurate predictions, the process usually involves large and complex datasets, which would add to computational costs for developing a model with good performance. Therefore, this study introduces the single-layer Transformer Convolutional Encoder algorithm (STCE), an improved version of the traditional transformer encoder. STCE is computationally efficient and does not compromise the accuracy of the prediction. In STCE, the fully connected Convolutional Neural Network (CNN) layer is used in the transformer to get the first temporal features and record long-range dependencies with Multi-Head Attention. To minimize complexity while maintaining performance, a single dense layer is used for the output instead of the Multi-Layer Perceptron (MLP) and omit positional encoding, which leverages the natural sequence order of the time series data. Additionally, since time-series price data normally comes with missing values, this study introduce a sequence nearest neighbor imputation algorithm for anchoring that data to complement the STCE method. This study focuses on various vegetable prices, such as tomatoes, long beans, and cucumbers, with empirical validation across various prediction prices, specifically 30-day, 60-day, and 90-day predictions. Predictions made with Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) show that the STCE algorithm is better than other deep learning algorithms, even the traditional transformer encoder. STCE algorithm not only has better performance, but it also reduces the computational time in the training with 12% fewer seconds compared to the transformer encoder and 22% fewer seconds for LSTM. This study not only provides valuable insights for farmers and planners in the agriculture market but also highlights the robust potential of transformer encoder in predicting commodity markets.