Improved RRT Path Planning Method Incorporating Deep Reinforcement Learning for Robotic Manipulator

The sampling randomness of the Rapidly-exploring Random Tree Star algorithm (RRT*) in complex 3D environments often leads to low planning efficiency and unstable path quality for robotic manipulator path planning. To address these challenges, this paper proposes an improved DDPG-guided RRT* algorithm (DDPG-RRT*) based on Deep Deterministic Policy Gradient. Firstly, we develop an end-node-based DDPG-guided sampling mechanism with a multi-objective reward function that integrates goal orientation and Gaussian obstacle constraints. This enhancement improves the goal-directed sampling distribution and strengthens exploration capability in narrow passages. Secondly, joint motion penalty and condition number penalty terms are incorporated into the reward function to ensure joint movement safety and singularity avoidance. Thirdly, a hybrid sampling strategy is implemented to probabilistically combine spatial random sampling with DDPG-guided sampling, achieving a balance between asymptotic optimality and enhanced planning efficiency and stability. Finally, simulation experiments demonstrate that the proposed DDPG-RRT* algorithm outperforms the RRT*, PRRT*, and RRT-Connect methods in terms of path length, number of nodes in the generated paths, computational efficiency, and smoothness of joint motion trajectories.