Environment-Adaptive Motion Planning via Reinforcement Learning-Based Trajectory Optimization

This paper proposes a novel environment-adaptive motion planning framework for mobile robots, which utilizes deep reinforcement learning to dynamically adjust optimization objectives according to various environmental and robot-ego characteristics, greatly enhancing the adaptability and robustness compared to existing motion planning strategies. Our approach features a two-stage trajectory optimization algorithm that optimizes for smoothness, safety, and efficiency-elements critical in practical applications. Firstly, we propose a reinforcement learning algorithm that dynamically adjusts optimization objectives based on the environmental context, which carefully encodes the environment, coarse initial path and robot information into the observation space. Additionally, two techniques are designed to reduce the sim-to-real gap: 1) integrating classical optimization framework as the motion planning backbone; 2) using low-dimensional input in the learning component, which minimizes discrepancies between simulated and real-world conditions. With the hybrid strategy, not only the interpretability and stability of the classical motion planning pipeline is preserved, but also the adaptive capability of emerging DRL techniques is fully utilized. The efficacy of the proposed solution is demonstrated through extensive simulations and real-world experiments, showcasing superior performance in terms of safety and efficiency across various testing scenarios. Especially in unknown and cluttered real office environments, our approach significantly improve safety (17% increase in success rate) and efficiency (29% reduction in time cost), compared to the popular traditional methods. (Supplementary video link: https://youtu.be/ph0pDGpI864) Note to Practitioners—This paper is motivated by the need for reliable and adaptable motion planning in various environments where mobile robots must navigate safely and efficiently. Traditional motion planning methods often struggle with the adaptability required in these settings due to their fixed optimization objectives and lack of flexibility. On the other hand, end-to-end deep reinforcement learning (DRL) approaches can adapt to changing environments but may suffer from catastrophic failures and a lack of interpretability. This paper introduces a hybrid approach that combines DRL with classical optimization techniques. The proposed method allows the robot to adjust its optimization objectives dynamically, enhancing its ability to handle unforeseen obstacles and changes in the environment. This is particularly beneficial in applications where robots must operate autonomously for extended periods without human intervention. The hybrid approach presented has shown promising results in simulations and real-world tests, demonstrating both stability and reliability.