Continuous State-Action-Observation POMDPs for Trajectory Planning with Bayesian Optimisation

Decision making under uncertainty is a challenging task, especially when dealing with complex robotics scenarios. The Partially Observable Markov Decision Process (POMDP) framework, designed to solve this problem, was subject to much work lately. Most POMDP solvers, however, focus on planning in discrete state, action and/or observations spaces, which does not truly reflect the complexity of most real world problems. This paper addresses the issue by devising a method for solving POMDPs with continuous state, action and observations spaces. The proposed planner, Continuous Belief Tree Search (CBTS), uses Bayesian Optimisation (BO) to dynamically sample promising actions while constructing a belief tree. This dynamic sampling allows for richer action selection than offline action discretisation. CBTS is complemented by a novel trajectory generation technique, relying on the theory of Reproducing Kernel Hilbert Spaces (RKHS), yielding trajectories amenable for robotics applications. The resulting trajectory planner kCBTS outperforms other continuous planners on space modelling and robot parking problems.