Information Relaxation and Dual Formulation of Controlled Markov Diffusions

Information relaxation and duality in Markov decision processes have been studied recently by several researchers with the goal to derive dual bounds on the value function. In this paper we extend this dual formulation to controlled Markov diffusions: in a similar way we relax the constraint that the decision should be made based on the current information and impose a penalty to punish the access to the information in advance. We establish the weak duality, strong duality and complementary slackness results in a parallel way as those in Markov decision processes. We further explore the structure of the optimal penalties and expose the connection between the optimal penalties for Markov decision processes and controlled Markov diffusions. We demonstrate the use of this dual representation in a classic dynamic portfolio choice problem through a new class of penalties, which require little extra computation and produce small duality gap on the optimal value.