A Q-learning based approach to design of intelligent stock trading agents

The portfolio management for trading in stock market poses a challenging stochastic control problem of significant commercial interests to finance industry. To date, many researchers have proposed various methods to build an automated portfolio management system that can recommend financial decisions for daily stock trades. However, most previous attempts to predict future returns of stocks have been largely unsuccessful due to the high complexity of integrating price prediction results with trading strategies. Motivated by this, this paper presents a new stock trading system, named Q-trader, that can overcome the limitations of the existing approaches through the joint optimization of the prediction results and the corresponding trading strategies. Specifically, the proposed framework employs four cooperative Q-learning agents that adaptively interact with their environments by use of feedforward neural networks. In order to achieve computational efficiency for global trend prediction, a novel data structure is proposed for compact representation of long-term dependencies of stock price changes. Experimental results on KOSPI 200 show that the proposed approach outperforms the trading systems trained by supervised learning both in profit and risk management.