Online and Robust Intermittent Motion Planning in Dynamic and Changing Environments

In this article, we propose RRT-Q
$^{\textrm{\,X}}_{\infty}$
, an online and intermittent kinodynamic motion planning framework for dynamic environments with unknown robot dynamics and unknown disturbances. We leverage RRT
$^{\textrm{\,X}}$
for global path planning and rapid replanning to produce waypoints as a sequence of boundary-value problems (BVPs). For each BVP, we formulate a finite-horizon, continuous-time zero-sum game, where the control input is the minimizer, and the worst case disturbance is the maximizer. We propose a robust intermittent Q-learning controller for waypoint navigation with completely unknown system dynamics, external disturbances, and intermittent control updates. We execute a relaxed persistence of excitation technique to guarantee that the Q-learning controller converges to the optimal controller. We provide rigorous Lyapunov-based proofs to guarantee the closed-loop stability of the equilibrium point. The effectiveness of the proposed RRT-Q
$^{\textrm{\,X}}_{\infty}$
is illustrated with Monte Carlo numerical experiments in numerous dynamic and changing environments.