Improved action and path synthesis using Gradient Sampling

Shortest paths generated through gradient descent on a value function have a tendency to chatter and/or require an unreasonable number of steps to synthesize. We demonstrate that the gradient sampling algorithm of [Burke, Lewis & Overton, 2005] can largely alleviate this problem. For systems subject to state uncertainty whose state estimate is tracked using a particle filter, we propose the Gradient Sampling with Particle Filter (GSPF) algorithm, which uses the particles as the locations in which to sample the gradient. At each step, the GSPF efficiently finds a consensus direction suitable for all particles or identifies the type of stationary point on which it is stuck. If the stationary point is a minimum, the system has reached its goal (to within the limits of the state uncertainty) and the algorithm naturally terminates; otherwise, we propose two approaches to find a suitable descent direction. We illustrate the effectiveness of the GSPF on several examples using the ROS and Gazebo robot simulation environment.