Financial Bidirectional Encoder Representations from Transformers with Temporal Fusion Transformer for Predicting Financial Market Trends

Nowadays, predicting financial market trends is a complex task due to market’s inherent volatility and complexity. The Machine Learning (ML) models capture temporal relationships to predict the market price movements. However, the existing model lacks integration of external sentiment and macroeconomic features which led to limit its capability to capture market-moving qualitative factors and broader economic context. Hence, this research proposes Financial Bidirectional Encoder Representations from Transformers with Temporal Fusion Transformer (FinBERT-TFT) to integrate external sentiment and macroeconomic features by capturing temporal relationships and external factors. Initially, the input data is collected from Yahoo Finance dataset and financial text data which is used for sentiment analysis. Then, the data is preprocessed with K-Nearest Neighbor (KNN) imputation to handle missing values and min-max normalization to normalize the input ranges respectively. After that, the FinBERT is employed to process the unstructured text data and generate daily sentiment scores for each stock. Finally, the TFT is used to predict the market trends by capturing quantitative patterns based on sentiment scores. From the results, the proposed FinBERT-TFT outperformed existing Convolutional Neural Network with Long Short-Term Memory (CNN-LSTM) in terms of Mean Squared Error (MSE) and Root MSE (RMSE) with 0.00004 and 0.00639 respectively.