A Cash Flow Based Framework for Asset-Liability Management with Deep Reinforcement Learning

While Reinforcement Learning (RL) has been successfully applied to financial portfolio optimization, current works focus on assets but rarely discuss the effect of liabilities. An iterative method is widely used in the transaction model, nevertheless its running time is not guaranteed. Also, debt-based assets which are commonly non-publicly traded, seem to be overlooked, and the difference between trading days and calendar days is usually ignored. These deficiencies limit the scope and depth of applying RL methods to the finance field. In this work, a cash flow based framework is proposed to deal with these issues. A modified algorithm with guaranteed running time is proposed for the transaction model during portfolio rebalancing. With the effect of debts being considered, this framework can be used to solve asset-liability management problems including optimizing financial portfolios. The PPO algorithm is also extended, adapted to a more general value function form which uses time-variant discount rates. The experiments include a toy example and a more complicated problem with real-world data. The theoretic discussion and the positive experiment results demonstrate that the cash flow based framework could work.