Simulated Annealing Monte Carlo Tree Search for Large POMDPs

Many planning and control problems can be modeled as large POMDPs, but very few can be solved scalably because of their computational complexity. This paper proposes a Simulated Annealing based on the Monte Carlo Tree Search for large POMDPs. The proposed algorithm determines an acceptance probability of sampling a back-propagation's outcome in the simulated annealing process. The experiments show that the proposed SAMCTS (Simulated Annealing Monte Carlo Tree Search) outperforms the original Simulated Annealing algorithm when applied to a large POMDP benchmark problem.