Improving the Cryptocurrency Price Prediction Performance Based on Reinforcement Learning

During recent developments, cryptocurrency has become a famous key factor in financial and business opportunities. However, the cryptocurrency investment is not visible regarding the market’s inconsistent aspect and volatility of high prices. Due to the real-time prediction of prices, the previous approaches in price prediction doesn’t contain enough information and solution for forecasting the price changes. Based on the mentioned problems in cryptocurrency price prediction, we proposed a machine learning-based approach to price prediction for a financial institution. The proposed system contains the blockchain framework for secure transaction environment and Reinforcement Learning algorithm for analysis and prediction of price. The main focus of this system is on Litecoin and Monero cryptocurrencies. The results show the presented system accurate the performance of price prediction higher than another state-of-art algorithm.


I. INTRODUCTION
T HIRD party institution for financial environments in a system of the traditional economy operates the payment in different forms. These environments are intermediaries between the fund-changing parties and, in the same way, controlling the complete transactions records. This process is working fine for limited money transactions, and it contains a lack of transparency, feasibility, security, and trust. To overcome these issues, there is a need to clear the intermediary parties from the final transactions. e.g., allowing the transaction of funds to be directly between parties can cause changes in terms of economic works. Cryptocurrency has the ability of currency exchange between groups or in a single way [1]. The network-based transactions and exchanges use the algorithm cryptographic for securing the transactions. The cryptocurrency is directly related to the blockchain framework and be devised the blockchain properties such as transparency and decentralization. Still, in the traditional cryptocurrency system, the central authority can't be controlled. This procedure validates the consensus algorithm of blockchain to solve the trust problem between the network stakeholders.
The cryptocurrency price is one of the researchers' inquis-itiveness in the entire world. The price changes are based on some factors like cost of the transaction, difficulty of mining, trends of markets, popularity, coins alternates, etc. [2]. The named factors give unstable price changes of a cryptocurrency over time and a difficult prediction process. Figure  1 presents the overview of cryptocurrency price prediction based on blockchain and machine learning techniques. The price data in cryptocurrency contains various categories: bitcoin, litecoin, ripple, monero, tether, and IOTA. In this process, we used two data sources as Litecoin and Monero. The machine learning section processes the data and slips it into a train and test set to prepare it for the blockchain network and feature extraction. The blockchain framework contains the transaction IDs, hash records, timestamps, and several blocks. The feature layer has reports related to cryptocurrency tokens, digital wallets, smart contracts, and distributed apps.

A. MOTIVATION
The main usage of cryptocurrency systems is for exchanging money. In the past years, cryptocurrency-based trading has become a pleasant topic [3]. This is the frequent idea of the professional stock market that it doesn't have a certain  place for enterprise regarding the dependency and volatility of the social sentiments. Besides this, the price prediction effectiveness of the different cryptocurrencies gives the ability of complete power computing of the blockchain [4]. The cryptocurrency value mainly affects many factors such as the past trend of prices, social sentiments, the volume of trades, and legislature. Motivated from the existing state-of-arts, we designed a model to predict the differences of cryptocurrency prices applying the Reinforcement Learning model. Similarly, the erratic fluctuation problem in the cryptocurrency price is addressed in this process. The main contribution of this process is divided as: • Applying Reinforcement Learning (RL) to present the prediction of Monero and Litecoin prices. • Performance evaluation of the presented system using the evaluation matrices e.g., MAE, MSE, RMSE, MAPE. • Using the blockchain framework to create a secure and transparent environment for price prediction.
The advantages of the presented system describes cryptocurrency characteristic which has the global access and it is easy to use for the medium type of transaction for storing wealth. Although, the different cryptocurrencies value is totally based on the social sentiments and trends of erratic market which has the less correlation with most of the financial assets. Based on this the traditional methods become ineffective for this process.
The rest organization of the paper is as follows. Section 2 presents the recent existing works regarding price detection. Section 3 presents the system model and definition of problem and formulation in cryptocurrency price prediction. Section 4 presents the prediction workflow and performance evaluation, and we conclude this process in the conclusion section.

II. RELATED WORK
A fundamental part of this section focus on the prediction and stability of the cryptocurrency price over decades. There are two main parts focused on in this research: cryptocurrency in blockchain and machine learning on the price prediction. We have explained the recent researches done in this area and the comparison of them.

A. CRYPTOCURRENCY IN BLOCKCHAIN
Blockchain technology applications go further in the peerto-peer system of payment. This gives the trust, privacy, and security to the system based on a distributed ledger to make the Internet of Things applications applicable for the system distributed storage [5]. The advantage of this system is to be decentralized and fully secure of whole environment which only allows that the new blocks append. The blockchain applications ranges are at the head of many blockchain and cryptocurrencies. Cryptocurrency is connected to blockchain because of providing an inducement to the machine and electricity consumption for blockchain validation. Cryptocurrency is the recent and new digital currency type for using the blockchain to increase transparency, immutability, and decentralization [6]. Regarding the increase of the usage of blockchain the cryptocurrency usage also increases. This contains the inherent value to this network based on various factors. This process is the new currency type that saves the values and increases the level of understanding price changes based on the values.   Figure 2 shows the price difference of cryptocurrency coins during years. The market capitalization from 2017 increased and summed to almost 19 billion USD. Regarding this amount, seven currencies and 97% of the market incorporated in 90% of the market capitalization. Figure 3 shows the market capitalization records based on year-wise and USD.

B. MACHINE LEARNING ON PRICE PREDICTION
Deep learning is a powerful machine learning algorithm for solving complex and nonlinear issues to exploit the huge number of data and good predictions. The price prediction with accurate results is a complex problem because the variation of values is a lot, but the deep learning approach overcomes this issue. Ji et al. [7] presented the comparison of Long Short Term Memory (LSTM) with Deep Neural Network (DNNs) and combined the results with the bitcoin price prediction. The presented result of their approach shows the acceptable accuracy of LSTM comparing with the other regression models. In this approach they tried to analyze the deep learning models in term of regression analysis which is immature to just for trading of bitcoin. Shintate et al. [8] presents the framework of trend classification and prediction for the non-stationary cryptocurrency time series data based on deep learning. The developed approach results show the LSTM model performance based on profitability analysis using the buy and hold strategy. The output results of this system show the LSTM generalized perfectly in the prediction of cryptocurrency prices. Peng et al. [9] proposed the Generalized Auto-regressive Conditional Heteroskedasticity (GARCH) with Support Vector Machine (SVM) for Bitcoin and Ethereum price prediction. The analysis was based on the low and high frequency and similarly the prediction is evaluating based on Dielbold-Matiano test and confidence set of Hansen's Model. Table 1 presents the summery of recent related studies regarding cryptocurrency price prediction.

III. SYSTEM MODEL AND PROBLEM FORMULATION
Cryptocurrency price prediction contains three important data sources. The first one is a market statistic. The second one is network information of blockchain that contains the transaction count and fee, hash rate, etc. The last one is google trends and volume of tweets. Most of the research works use the mentioned data sources to apply to their model which is mostly regression models. The data collected from three sources were aggregated to the data loader. Data features become normalized, and in the next step, create the data stacks. Reinforcement learning process the input data for predicting the N days price. Figure 4 presents the price prediction process in detailed form.   Table 2 presents the list of acronyms which used in this work.

A. DATA
In this process, Litecoin and Monero are considered as data sources of cryptocurrency price prediction. The training set for Litecoin contains the data from 2016-2020 with 1276 points of data records. The trained model is supposed to predict the next day's price. The training set Monero is from 2015 to 2020. The total number of records of data is 1850. The trained model predicts the next-day price records. The training set for Litecoin and Monero is 80% and the test set is 20% for processing and analyzing.

1) Transactions and Market Volume
Regarding the days, the number of transactions performed. The share market is not part of the stock exchange because there is no timetable for opening and closing. The number of traded coins are records of a single day, and it's one of the features. The market volume is specified for the particular day, and it supposes to have the units of currency. The coins' quantity is the index of encompassing value.  Mining the single block coins required the mining difficulty. The transactions confirmation requires the specific hardware to get more hash for mining a block. Power consumption and hash rate have a trade-off in between. This process shows the profit of mining rate and difficulty. The minor useful income opposed the use of power-consuming resources and time. Increasing the number of minors decreases the reward of minors exponentially.

3) Confirmation Time and Capitalization of Market
Transaction confirmation between parties requires the average time and logging to the block table. The transaction logging needs around ten minutes to confirm the transaction, which is based on the activated users and their location for table block update. The cryptocurrency amount per day is based on USD.

4) Google Trends and Tweets
Increasing the volume of tweets cause more transaction from people. This process is not only the correlation it contains the feedback loop too. Tweet volume is not the end of correlation, and people tend to look for updated trending topics [20]. Google search spike also has speculation to be in relation with coins price that is an insignificant assumption.

B. REINFORCEMENT LEARNING-BASED PRICE PREDICTION
Reinforcement Learning (RL) is one of the useful machine learning frameworks for evaluating the reward selected from the action. RL is based on a model-based and model-free design which gives the learning-based structure to the system. Training the learning-based model can improve the designed system structure and improve the system performance with high accuracy. RL's key goal is to determine a policy for actions to take the various states and maximize the rewards in the future. In some cases, the RL algorithm performing this regarding the learning from reward to take the certain action and specify the policy for the valuable actions. In the process of price prediction, Figure 5 presents the prediction steps based on usage of the RL algorithm and the collected data regarding Litecoin and Monero. The raw data information contains four steps of pre-processing, feature engineering, transformation, and feature selection. We split the dataset and applied it into the RL procedure to train and test the price information. Table 3 gives the information related to extracted features from the price prediction system. In total, there are 13 extracted features with explanations and units.

C. EVALUATION METRICS
The trained price prediction model was evaluated • Root Mean Squared Error (RMSE): • Mean Square Error (MSE):

IV. IMPLEMENTATION AND RESULTS
This section defines the implementation process and development area of the price prediction system in detail. Table 4 summarizes the components of machine learning and blockchain framework in this system. The operating system of the machine learning environment is windows 10. Browser is Internet Explorer, Firefox, and Chrome. The programming language for train and test the developed models are Python and IDE. The blockchain framework was designed based on the Ubuntu Linux 1804 LTS operating system and Node.js programming language. The CPU model is Intel(R) Core(TM) i7-8700 @3.20 GHz, and the docker engine version is 18.06.1-ce. Docker composer version is 1.13.0, and the IDE is Composer Playground. The memory usage in this system is 12GB.

A. PRICE PREDICTION DAILY ANALYSIS
Daily analysis of Litecoin and Monero presented in Figure 6 and 7. The trained data for Litecoin and Monero is from the 2016 to 2020 time period and contains 1272 points of data. The trained model is to predict the 3 days price from March 22 to 24. The price records are in USD.We have compared our proposed approach result with other existing works. The yellow color in the figure defines the existing works [21] related to this system. The orange color shows the proposed system results, and the red color presents the actual value VOLUME 4, 2016

B. PRICE PREDICTION WEEKLY ANALYSIS
The weekly analysis results in this system are summarized in Figure 8 and 9 for Litecoin and Monero during one week and seven days in the price of USD. The prediction is based on the step size and RMSE. The colors present the differences of the existing study, proposed Litecoin and Monero, and actual value. The yellow color shows the records of existing works. The orange color shows the records of the proposed Litecoin and Monero, and the red color shows the actual value in both of them.

C. PRICE PREDICTION MONTHLY ANALYSIS
The 30 days price prediction results of Litecoin and Monero are presented in Figure 10 and 11. It can be visible in both Figures the actual value and the predicted value are close together, and the direction of the trend is also highly stable.     Table 5 shows the results due to three days. Table 6 shows the results of seven days and Table 7 shows the results of 30 days.

V. CONCLUSION
Cryptocurrency price prediction is a challenging area for researchers regarding the external and objective factors that affect price prediction, such as ARIMA, SARIMA, etc, which are normally used for financial schemes analysis. The mentioned are mostly used for time-series but contain lots of limitations regarding the assumptions. In the recent research topics, usage of neural networks contains acceptable results with many variants regarding the price prediction topic. In this paper, we defined the Reinforcement learning prediction approach integrated with blockchain framework for price prediction of Litecoin and Monero. The proposed scheme shows better performance comparing with other state-of-art in this area. In this system, we achieved to the higher accuracy comparing with the other systems in term of Litecoin and Monero which we earlier discuss in related work. The goal of this research is to achieve the better performance for the prediction of cryptocurrencies with less error rate.  He is currently an Associate Professor with the Computer Engineering Department, Jeju National University. His research interests include AI and machine learning, pattern recognition, blockchain and deep learning-based applications, big data and knowledge discovery, time series data analysis and prediction, image processing and medical applications, and recommendation systems. VOLUME 4, 2016