Reliable Path Planning Algorithm Based on Improved Artificial Potential Field Method

In order to solve the “minimum trap” of Artificial Potential Field and the limitation of traditional path planning algorithm in dynamic obstacle environment, a path planning algorithm based on improved artificial potential field is proposed. Firstly, a virtual potential field detection circle model (VPFDCM) with adjustable radius is proposed to detect the “minimum trap” formed by the repulsion field of obstacles in advance. And the motion model of unmanned vehicle is established. Combined with the improved reinforcement learning algorithm based on Long Short-Term Memory(LSTM), the radius of virtual potential field detection circle is adjusted to achieve effective avoidance of dynamic obstacles. The reliable online collision free path planning of unmanned vehicle in semi closed dynamic obstacle environment is realized. Finally, the reliability and robustness of the algorithm are verified by MATLAB simulation. The simulation results show that the improved artificial potential field can effectively solve the problem of unmanned vehicle falling into the “minimum trap” and improve the reliability of unmanned vehicle movement. Compared with the traditional artificial potential field method, the improved artificial potential field method can achieve more than 90% success rate in obstacle avoidance.


I. INTRODUCTION
The development of information technology has promoted the industrial transformation, accelerated the arrival of the intellectualized era, and led to the emergence of smart factories, such as unmanned automobile factories, Park unmanned vehicle cargo handling, indoor unmanned vehicle logistics sorting. Therefore, unmanned vehicle technology plays an important role in smart factories [1], [2]. Although unmanned vehicle is widely used in smart factories, with the diversity and complexity of unmanned vehicle application scenarios, it poses a huge challenge to the development of unmanned vehicle technology. For example, the randomness and instability of obstacles in different scenarios will directly affect the efficiency of unmanned vehicle, so it is important to solve the unmanned vehicle routing problem in every dynamic unknown environment.
The application fields of path planning are very extensive, such as path planning of robot manipulator, aircraft path The associate editor coordinating the review of this manuscript and approving it for publication was Yun Lin . planning, cruise missile path planning, Traveling Salesman Problem (TSP) and its derivative vehicle (VRP) path planning, virtual assembly path planning, path planning based on road network, electronic map GPS navigation path search and planning, routing problems, etc. At present, the main global path planning algorithms [3] are Dijkstra algorithm [4], A-star algorithm [5], ant colony algorithm [6], genetic algorithm [7], and reinforcement learning [8], [9]. Global path planning algorithm needs to master the global environment information, and then make path planning based on the environment information, which lacks the adaptability to unknown environment. Local path planning [10] algorithms, such as Dynamic Window Approach (DWA) algorithm [11], artificial potential field [12], etc. Local path planning algorithms are prone to fall into local optimum because they cannot guarantee the optimal solution. And most of the above methods are applicable to static environments, not dynamic environments. Artificial potential field method has simple algorithm, few iterations, strong real-time performance and smooth path, but it is easy to fall into local minimum and target unreachable problems. In view of the ''minimum trap'' in the artificial potential field, scholars at home and abroad have proposed solutions such as adding additional force, improving the force field, and introducing other algorithms.
Therefore, in order to achieve unknown dynamic environment for unmanned vehicle routing, an improved artificial potential field is proposed to achieve local online path planning, and the Q-Learning algorithm of LSTM is combined to avoid dynamic obstacles. This paper innovatively presents a VPFDCM, which detects the ''minimum trap'' [13] based on the distribution and quantity information of the virtual potential field detection points on the circle contacting the repulsive field of obstacles, and effectively avoids it. In addition, the detection point information can be used as the information input of LSTM. By using the characteristics of LSTM processing time series data, the predicted state space of obstacles can be obtained to achieve collision-free path planning for unmanned vehicle in unknown dynamic environment.
In general, the artificial potential field has been widely used in the path planning of ground mobile robots, channel path planning and underwater path planning.

II. ARTIFICIAL POTENTIAL FIELD METHOD
Artificial potential field is a virtual force method proposed by Khatib in 1986. As a classical path planning algorithm, it is widely used in local path planning. It is a method that simulates a potential field manually in the working scene of a robot, which is analogous to the electromagnetic field. The robot is affected by attraction and repulsion, and the path is planned according to the characteristics of the potential field. As shown in Fig. 1. If a robot wants to move from one location to another, it can add an attraction field to the target location. Similarly, to avoid obstacles, it needs to add a repulsion field attenuating with distance around the obstacle, giving the robot a tendency to avoid obstacles. This establishes an artificial potential field along which the robot travels until it reaches its target point. As shown in Fig. 2.
Where, F Target is the attraction of the robot by the target point, and F Obstacle is the repulsion of the robot by the obstacle. The robot moves along F Total. The attractive potential field function of the artificial potential field method is generally set to: where, k a is a gravitational gain factor. X g = x g , y g is the location of the target point, and X n = (x n , y n ) is the location of the robot. The size of the attractive potential field is shown in Fig.3. In Fig.3, the coordinate point (50, 50) is the location of the target point. It can be seen from the graph that the farther the robot is from the target point, the larger the value of the attractive potential field. The attraction of the target point to the robot is a negative gradient of the attractive potential field function: The repulsive field function of an obstacle is generally: where, k r is a repulsion gain factor, and r 0 is the maximum influence range of the repulsive potential field generated by the obstacle. X g = x g , y g is the location of the target point, and X n = (x n , y n ) is the location of the robot. The size of the repulsion potential field is shown in Fig.4. In Fig.4, the coordinate point (150, 150) is the location of the obstacle. As you can see from the graph, the repulsive potential field established by the obstacle has a certain range of influence and only has potential energy in a certain area. The closer the robot is to the obstacle, the greater the potential energy of the robot. When a robot moves toward an obstacle, repulsion prevents the robot from approaching the obstacle. The robot moves toward a position with less potential energy and moves around the obstacle, thus completing the action of avoiding the obstacle. Repulsion is the negative gradient force of a repulsive potential field whose mathematical expression is: It can be obtained that the force exerted by the artificial potential field on the robot is: The potential field model of the robot motion direction is shown in Fig.5.
From the potential field model in the figure above, you can see that the mountain is the location of the obstacle, that is, near the repulsion field, and the valley is the location of the target, that is, near the attraction field. The direction of motion of the robot is from high to low.
Compared with other classical obstacle avoidance algorithms, artificial potential field method has many outstanding advantages, such as: small computation, solving local obstacle avoidance problems, solving sudden threats, and so on. Therefore, this algorithm is widely used in obstacle avoidance algorithms.
However, the artificial potential field method has obvious drawbacks. The robot relies on the overlap of potential fields detected from all directions to obtain the coincidence field, and the direction and size of the coincidence field [14] are used to judge the next trajectory. But if the coincidence field approximates zero, the robot will not move and will stop. The more obstacles there are in the area, the higher the probability that the coincidence field will be zero, and the easier it will be to stagnate. This phenomenon is called the ''minimum trap''. In this article, solving the ''minimum trap'' is one of the main topics discussed.

III. IMPROVED ARITIFICIAL POTENTIAL FIELD METHOD
The robot avoids obstacles and moves to the target point under the combined force of repulsion and attraction potential fields.
In the following two situations, the robot will fall into the ''minimum trap''.

1) EXCESSIVE REPULSION
When multiple obstacles are between the robot and the target point, multiple obstacles exert repulsion on the robot. Because the robot repulsion at this point is too large, the overlap of multiple repulsions exactly equals the size of gravitation, resulting in the robot falling into the ''minimum trap'' and stopping moving at this point. As shown in Fig.6.  The potential field diagram of the ''minimum trap'' due to excessive repulsion is shown in Fig. 7. The robot stops moving with zero total force.

2) FORCE BALANCE
When the obstacle is between the robot and the target point, and the three are in the same straight line, the attraction and repulsion are collinear but in the opposite direction. The robot will approach the obstacle and the target point simultaneously under the action of force. At some point, the resultant force may be zero or the resultant direction may change in the opposite direction. In this case, the robot may stop or oscillate back and forth because it is trapped in a ''minimum trap''. As shown in Fig.8. The potential field diagram of the ''minimum trap'' due to force balance is shown in Fig. 9. The robot stops moving or oscillates when the total force is zero.
In cases where the path planning fails due to the above problems, based on the traditional artificial potential field method, a VPFDCM is presented in this paper. The circle consists of N virtual detection points centered on unmanned vehicle k and radius R to form a virtual detection circle T . Barrier repulsion fields can be detected at each point, and as the unmanned vehicle approaches the barrier, the number of points that circle T touches the barrier repulsion field, n, is normally distributed, as shown in Fig. 10.  Note: d indicates the distance between the unmanned vehicle and the obstacle in meters; n denotes the number of points, that a circle T touches the repulsive field of an obstacle, in units.
When two discrete normal distributions occur on the detection circle T and the calculated distance d 1,2 between the obstacles is less than the safe distance d safe , the ''minimum trap'' can be predicted, as shown in Fig. 11. d safe is the distance between two obstacles that may form a minimum trap. The conditions to be met to form the ''minimum trap'' are shown in (7).

1) ESTABLISHING KINEMATICS MODEL OF UNMANNED VEHICLE
The artificial potential field method is applied to the route planning of unmanned vehicle. In order to ensure the stability of the unmanned vehicle motion, the kinematic constraints VOLUME 10, 2022  of the unmanned vehicle itself need to be considered. The bike kinematics model [15] is used to simplify this process, as shown in Fig. 12.
Simplified formula (8): where, δ is the turn angle of the front wheel, L is the pitch, R is the radius of the circle for the unmanned vehicle to move under this turning angle. Equations consisting of radius of curvature R, angular velocity ω, and velocity ν: ν = ωR, the kinematic model of the unmanned vehicle can be obtained as shown in formula (9): where, ν is the speed of the unmanned vehicle, · x is the speed of the unmanned vehicle in the X-axis direction of the world coordinate system, · y the speed of the unmanned vehicle in the Y-axis direction of the world coordinate system, θ is the heading angle of the unmanned vehicle in the world coordinate system, and · θ is the angular speed of the unmanned vehicle.

2) UPDATING HEADING ANGLE
The two outer contact points between the detection circle and the repulsion fields of the two obstacles are set to a 1 , b 1 . The two points form two angles α, β with unmanned vehicle k and target point X , and the minimum angle is used as the updated heading angle of unmanned vehicle k, as shown in Fig. 13. That is, θ = min (α, β). This leads to a shorter path away from the ''minimum trap''.

IV. IMPROVED ENHANCED LEARNING DYNAMIC ENVIRONMENT PATH PLANNING ALGORITHM BASED ON LSTM
A. LSTM LSTM [16] is a long-term and short-term memory network developed from RNN (Recurrent Neural Network) [17]. Compared with RNN, LSTM has a good memory ability and has a significant role in solving the problem of gradient disappearance and long dependence that traditional RNN cannot solve. Therefore, LSTM has become a widely used neural network [18].
The classical LSTM formula is Among them (10) is the forget gate, (11) (12) is the input gate, (13) is the output gate. Fig. 14 below can be drawn from the top. Compared with traditional RNN, the biggest feature of LSTM is the addition of C t−1 and C t , which is called the cell state. Cell states are updated once during each iteration, preserving the best results from previous iterations to form long-term memory. In this way, the long history information of the past can be used to predict the future situation [19]. In combination with the short-term last information, a long and short memory network is formed, which improves the learning ability of the robot to avoid obstacles [20], [21], [22]. It avoids the influence of short-term memory bias on the whole network and improves the anti-jamming ability of the system.

B. Q-LEARNING ENHANCED LEARNING ALGORITHM WITH LSTM
Although the improved artificial potential field method can effectively avoid the ''minimum trap'', it has poor adaptability in unknown dynamic environment. The reason is that the VPFDCM cannot predict the movement state of obstacles, and for obstacles that move faster, the model can easily cause information loss. LSTM can better handle time series tasks. To solve this problem, the LSTM loop neural network combined with Q-Learning's intensive learning algorithm structure is proposed to optimize the VPFDCM.
First, the LSTM loop neural network is used to process the change information of the repulsion field of the obstacle returned by the VPFDCM, then the LSTM algorithm is used to generate the output of the current time, the output of the current time refers to the characteristics of the change trend of the information before the time [23], to predict the movement state of the obstacle, to get the prediction state space S 0 of the obstacle further. The radius R of the circle model detected by the virtual potential field is adjusted to track the dynamic obstacles. The steps are: Step 1: Construct a VPFDCM with adjustable radius R.
Step 2: Obtain contact point n distribution and quantity information as input of LSTM.
Step 3: Obtain the predicted state space S 0 of the obstacle.
Step 4: Adjust the radius R by combining the spatial location of the predicted state space S 0 with the unmanned vehicle state. Fig. 15 above is an LSTM Q-Learning structure diagram. The steps to improve the artificial potential field method are shown.
The obstacle prediction state space S 0 obtained is added to the unmanned vehicle state space S in Q-Learning, and the reward function R is reset to enable the unmanned vehicle to avoid dynamic obstacles with the best reward.
where, d O 1 k is the distance between the obstacle and the unmanned vehicle, r is the radius of the repulsive field of the obstacle, R is the radius of the circle model for the virtual potential field, and d a 1 a 2 is the distance of the intersection of two circles.
where, ν · t represents that the car continues to travel at speed ν along the current heading angle. This paper adds the prediction status of Unmanned Vehicle and resets reward function [24]. When the distance between the unmanned vehicle and the obstacle center is greater than the unmanned vehicle prediction distance, reward =−1 is set to give a positive return. Conversely, it predicts a ''collision'' and gives a negative return to change the direction of the unmanned vehicle away from the prediction area.

A. ROUTE PLANNING ALGORITHM FLOW
The flowchart of the improved artificial potential field path planning algorithm is shown in Fig. 16.

B. EXPERIMENTAL ENVIRONMENT AND MODEL BUILDING
This article uses MATLAB to build a 400 × 400 2D map. The number of obstacles N O = 45 is randomly set in space, and the influence range of the repulsion field of a single obstacle is set r = 5. A random disturbance factor is added to each obstacle to make the obstacle move in an irregular motion mode. Fixed and moving ''minimum traps'' have been added to facilitate observation of the results. A VPFDCM is constructed. The function relationship between the number of points detected N and the radius R is N = 1 2 πR , as shown in Fig. 10, and the points are evenly distributed around the circle. The simulation results are shown below, in which point (10,10) is the starting point of the unmanned vehicle. Point VOLUME 10, 2022 As can be seen from the above figure, starting from its starting point, the robot moves toward its target point, which is gravitated by the target point and repulsed by surrounding obstacles. Fig. 17 is a traditional artificial potential field method in which the robot oscillates back and forth when it encounters a ''minimum trap''. Failure to reach the destination occurs. Fig. 18 shows an improved artificial potential field method after adding a VPFDCM. It can be seen that the robot is facing a ''minimum trap'', which is perceived ahead of time and avoided effectively. When it is predicted that the robot will fall into the ''minimum trap'', the robot changes its heading angle to escape it. The validity of this method is verified.

D. COMPARISON BASED ON LSTM Q-LEARNING ARTIFICIAL POTENTIAL FIELD METHOD
Placing the VPFDCM with fixed radius R in the environment of dynamic obstacles, it is found that the robot collides with obstacles that move irregularly or at high speed without time to avoid them. In view of this situation, this paper uses LSTM Q-Learning algorithm to propose a new VPFDCM  with adjustable radius R to track and predict the spatial state of obstacles, so as to avoid effectively.
As shown in Fig. 19, when a random disturbance factor is added to the obstacle, the robot will not be able to avoid and collide with the obstacle which moves faster. After many experiments, the probability of collision is about 78%. As shown in Fig. 20, the improved model shows that the robot can avoid high-speed obstacles and has a more obvious predictive behavior in the route. Compared with the previous algorithms, the improved algorithm has better adaptability in dynamic environment. The feasibility of this method is verified.   In Table 1, after the coordinates of the detection point on the model coincide with the coordinates of the repulsion field of the obstacle, the second coordinate information obtained does not conform to the motion of the robot and the obstacle. The success rate of avoidance is based on whether the robot makes the avoidance action and observes whether the avoidance is successful or not. From the data in Table 1, we can see that the improved model proposed by LSTM Q-Learning algorithm has a better improvement.
The simulation results show that the traditional artificial potential field method uses the interaction between the repulsion field and the gravitational field to control the robot. When the robot encounters a minimum trap, it often appears ''stuck'' and oscillation. By adding a VPFDCM, the robot can perceive obstacles ahead of time and avoid the minimum trap effectively.
At the same time, in the dynamic environment, the VPFDCM with fixed radius R cannot accurately avoid obstacles with irregular movement and high-speed movement, or even collide, and some information is missing. The LSTM Q-Learning algorithm is used to predict the motion state of the obstacle, and the radius R of the VPFDCM is set to be adjustable so as to follow the obstacle to a certain extent and avoid missing information. The results show that the robot can successfully avoid dynamic obstacles and achieve collision-free path planning. The validity of the algorithm is proved.

VI. CONCLUSION
To solve the ''minimum trap'' problem of traditional artificial potential field, a VPFDCM is proposed to avoid the ''minimum trap'' in advance. For dynamic unknown environment problem, LSTM Q-Learning algorithm is incorporated, and a virtual potential field with adjustable radius is proposed to detect circle model to sense the location of dynamic obstacles and predict their movement trend.
The simulation results show that the proposed method can effectively avoid the ''minimum trap'', as shown in Fig.18. And in the obstacles of movement, it can avoid obstacles and reach the end point smoothly. The effectiveness and robustness of the algorithm are proved.
The artificial potential field method is suitable for application in the changing environment. This method is conducive to the bottom, online control of mobile robot motion, and it is simple to construct. It is more and more widely used in the smooth control of moving routes and online collision avoidance with obstacles. Therefore, the improved method in this paper is expected to be applied to cargo sorting in indoor unmanned vehicle dynamic obstacle environment and autonomous route planning of field unmanned vehicle. If two-dimensional information can be used to achieve threedimensional, it also has the potential to be applied to the path planning under the dynamic obstacles in the air of the UAV.

ACKNOWLEDGMENT
Jie Luo would like to thank Prof. Wang Zhongxun (his mentor) who has put forward valuable comments on the direction of topic selection, the structure of the article, and the content of the article. Then, Jie Luo thanks his brother for his help in deriving the formula of this article, who has put forward valuable comments on the direction of topic selection, the structure of the article, and the content of the article, and for his help in learning the theoretical knowledge and deriv- VOLUME 10, 2022 ing the formula of the article. Second, Jie Luo thanks his laboratory classmates for helping solve all kinds of problems in the process of writing my article.