Pseudo-Target Optimization Strategy Based on Policy Iteration Algorithm

With autonomous robots becoming increasingly integrated into human society, the efficiency of their path optimization is of paramount importance. To address the issue of redundant states in the application of the policy iteration algorithm in the environmental model optimization process and to accelerate the convergence speed of the algorithm, this paper proposes a pseudo-target optimization algorithm based on the policy iteration algorithm in combination with the Manhattan distance. The proposed algorithm prioritizes updating the surrounding state policy centered on the pseudo-target state, the set composed of these states is called a local state set. Once the state policies within the aforementioned set have converged, the next pseudo-target state is identified through the use of the Manhattan distance, and the reward distribution is subsequently changed. Furthermore, the objective of quantifying the inflection point can be accomplished by regulating the length parameter n of the local state set, which corresponds to the different steering characteristics of robots in practical applications. Finally, the superiority of the algorithm is validated through experimental simulation. The optimized algorithm increases the convergence speed by more than 50 % when the parameter n is small, and gradually approaches the convergence speed before optimization when the parameter n is large. The maximum difference in the number of inflection points corresponding to different n values is nine. With the efficiency inversely proportional to the inflection point, this enables a balance between the steering characteristics of autonomous robots and the convergence speed of the algorithm in practical applications.