A New Method for Mean-Variance Optimization of Stochastic Dynamic Systems

In this paper, we propose a new optimization method to simultaneously maximize the average return and to minimize the reward variance of a stochastic dynamic system. This problem cannot be formulated as a standard Markov decision process (MDP) since the optimization criterion is a combined metric with mean and variance. Traditional methods, such as dynamic programming, are not valid. We resort to the sensitivity-based optimization theory and propose a new method to solve this problem. We derive a performance difference formula which quantifies the difference of the mean-variance combined metrics under any two different policies. Some optimality structures of this problem are also derived, which can be utilized to further develop iterative algorithms to optimize the mean-variance metrics.