Optuna Based Optimized Transformer Model Approach in Bitcoin Time Series Analysis

Crypto markets present significant challenges in financial time series forecasting with their high volatility and unpredictable nature. In this study, Optuna Based Optimized Transformer (OBOT) was proposed for time series forecasting for Bitcoin, the pioneer of cryptocurrency markets. To compare the proposed OBOT, Autoregressive Integrated Moving Average (ARIMA), Gradient Boosting Trees, Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Temporal Convolutional Network (TCN) with optimized hyperparameters were used. In particular, after the success of Transformer models in natural language processing, studies have been conducted on their potential for time series problems. The models were evaluated using Optuna for hyperparameter optimization and their performance was compared with Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) metrics. The generalized performance of the models was tested by dividing the data set into different time steps (6–12) and training and test sets at different rates (0.5-0.5, 0.7-0.3, 0.8-0.2). The results show that the proposed OBOT approach stands out for Bitcoin with an RMSE value of 0.0079 and a MAE value of 0.0122. These findings reveal that the proposed OBOT approach have significant potential in crypto market forecasting and should be examined in more detail in future studies.