Receding-horizon planning using recursive Monte Carlo Tree Search with Sparse Action Sampling for continuous state and action spaces

This paper introduces a recursive, sampling-based Monte Carlo Tree Search (MCTS) approach to planning, i.e. receding horizon control, in continuous state and action nonlinear set-point control problems. Usually, predictive control methods for nonlinear systems aim at (sub-)optimal control behavior through solving a dynamic optimization problem over a fixed prediction horizon in every time step. Tree-based methods, which have recently gained interest in planning, model-based reinforcement learning and path planning, replace the optimization problem by an incremental probabilistic search for (sub-)optimal open-loop control sequences in the space of variable-length closed-loop trajectories. The benefit of using a weaker optimization procedure is that the algorithm is very simple to understand/apply and works for general nonlinear systems. This article is concerned with increasing the control performance and sampling-efficiency of MCTS in continuous state and action spaces based on ideas from the field of standard nonlinear model predictive control.