A Deep Multimodal Reinforcement Learning System Combined with CNN and LSTM for Stock Trading

This paper proposes a deep multimodal reinforcement learning policy combining convolutional neural network (CNN) and long short-term memory (LSTM) neural network for stock price prediction. The proposed machine learning system generates various charts from stock trading data and uses them as inputs to the CNN layer. The features extracted through CNN layer are divided into column vectors and input to the LSTM layer. In order to utilize various types of charts, a multi-modal structure is used in which the input layer and the hidden layer are separated, and the individual operation results are combined. For reinforcement learning, we define agents’ policy neural network structure, reward, and action, and provide buying, selling, and holding probabilities as final output. In order to verify the performance of the proposed policy, we conducted training and testing with the daily data of the 256 stocks listed on the Korea KOSPI. Data from March 2009 to March 2019 were used as training data. When April 2019 data was used as test data, the KOSPI rose 0.5%, but the proposed machine learning model achieved a profit of at least 0.89% up to 3.06%. When the data for May 2019 was used as a test, the KOSPI fell by 7.8%, but when invested using the proposed machine learning model, the loss rate ranged from a minimum of 0.82% to a maximum of 6.81%. Therefore, we can see that performance is excellent both in bear and bull market.