Integration of Cell-Mapping and Reinforcement-Learning Techniques for Motion Planning of Car-Like Robots

The aim of this work has been to integrate the Cartesian space together with the kinematics and dynamics spaces of a car-like robot. We propose a new algorithm that obtains a minimum-time solution to the optimal motion planning of the vehicle. The new algorithm is based on the combination of cell-mapping and reinforcement-learning techniques. This algorithm can obtain the environment and vehicle parameters from received experience without needing a mathematical model. The algorithm uses a transformation of the cell-to-cell transitions to reduce the time that is spent in the knowledge of the vehicle dynamics and environment. Four state variables have been considered: 1) the velocity of the vehicle; 2) the $x$ Cartesian coordinate; 3) the $y$ Cartesian coordinate; and 4) the orientation of the vehicle. In addition, two different control actions can act on the vehicle: 1) the traction torque that was used for speeding up/braking the vehicle and 2) the steering angle. The results show the applicability of the proposed algorithm in environments with the presence of obstacles.