Dynamic Resource Reservation Based Collision and Deadlock Prevention for Multi-AGVs

Automated guided vehicles (AGVs) are widely used for material handling in warehouses and automated production lines due to their high efficiency and low cost. However, AGVs usually interact with each other because of the restricted capacity of the layout. Although many algorithms have been proposed to address the problem, most of them are inefficient for collision and deadlock avoidance in dynamic environments. This paper proposes a dynamic resource reservation (DRR) based method supporting time-efficient scheduling and collision avoidance of multiple AGVs. In this method, the layout is divided into square blocks with the same size that are abstracted as points in the undirected graph. In order to solve the collision and deadlock problem dynamically, the shared resource points of each vehicle are extracted from their guide paths in real time. Unlike the traditional approaches most of which adopt a static point occupation policy, DRR exploits dynamical reservations of shared resource points to change AGV movement states for avoiding collisions and deadlocks, resulting in better time efficiency. We jointly implement the algorithm on both central and local controllers. Extensive simulation results demonstrate the feasibility and efficiency of the proposed collision and deadlock prevention method.


I. INTRODUCTION
In recent years, automated guided vehicles (AGVs) gradually play a more and more important role in warehouses and industries [1]. Introduction of automated guided vehicles (AGVs) for material handling and visual grab in warehouses reduces operation cost and improves the throughput and efficiency of goods transportation [2], [3]. However, AGVs very often interact with each other during movements. It is necessary to use an effective on-line supervisory control strategy to solve collision and deadlock problems in the system layout.
Algorithms for dispatching, routing, scheduling, collision and deadlock avoidance can be centralized or decentralized. [4]. In the decentralized approaches, agents can obtain information generally from their neighbors. The main advantages of such a solution are its reliability, flexibility and scalability, but its security and efficiency are relatively low [5]. Digani et al. [6] propose a hierarchical traffic control algorithm, which implements path planning on a two-layer architecture. The first layer is a topological graph of the The associate editor coordinating the review of this manuscript and approving it for publication was Shaohua Wan . plant where each node is a macro-area of the environment. The second layer is the real route map on which the AGVs move. The coordination among the AGVs is obtained by exploiting local negotiation (i.e. decentralized coordination). Zaremba et al. [7] present a new approach for distributed control of automatic guided vehicle systems which uses max-algebra formalism to model system operations. Its novelty is that it replaces the classical problem of vehicle real-time scheduling with the problem of distributed time setting. Draganjac et al. [8] propose a strategy for coordinating vehicles in multi-AGV systems, which integrates autonomous motion planning and decentralized decision making. The coordination algorithm ensures reliable resolution of different conflict situations among the vehicles based on a removal strategy and a private zone mechanism. Cancemi et al. [9] propose a distributed method to coordinate heterogeneous vehicles in a large-scale industrial environment. The method consists of a resource-sharing protocol and a re-planning strategy. Based on the distributed resource-sharing protocol and the re-planning strategy, vehicles use the map information to calculate the paths and coordinate the motions. Roszkowska [10] presents a distributed control mechanism for coordinating heterogeneous mobile robots. The method prevents collisions and deadlocks by reducing the speed of some robots. In [11], the authors propose a shared resource based decentralized coordination algorithm for safe and efficient managements of a group of mobile robots. Pérez et al. [12] propose a decentralized method based on hierarchical path planning and mutually exclusive resource for coordination. When a vehicle dynamically plans its own trajectory, the other vehicles are treated as static obstacles. In [13], the authors propose a distributed predictive path following controller with arrival time awareness for multiple waterborne automated guided vessels. The work presented in [14] describes a distributed control system for coordinating vehicles moving along a network of interconnected predefined path areas. The efficiency of the system has been demonstrated with laboratory robots. However, since the vehicle is not autonomous but controlled by area controllers, this approach requires additional infrastructures to be installed in the work environment.
In the centralized control approaches, all information related to AGVs and the states of the transportation systems are stored and computed in central controllers. The centralized systems generally have high efficiency, but incur high time complexity. The methods based on time windows to avoid collisions and deadlocks are currently in use. In [15], the authors use time windows to solve the shortest path problem dynamically. Since the number of active vehicles and the corresponding missions change with time, the proposed routing method makes the determined shortest path feasible by time window elongation, resulting in collisionand-deadlock-free travelling for all active vehicles. However, this method is usually used in topologies with unidirectional lanes. Zhang et al. [16] propose a collision-free routing method. The route is predetermined by the improved Dijkstra algorithm. Four alternative methods are proposed to avoid each type of collision. Fan et al. [17] propose a heuristic algorithm to search for free time windows of the blocks in the selected paths for each AGV, and select the path with the earliest arrival time window as its scheduled path. In dynamic environments, the calculation of time windows subjects to many factors, such as acceleration and deceleration time of AGVs, external obstacles (e.g. people), which makes it difficult to calculate the time windows accurately. Imprecise time windows and external obstacles can lead to unpredictable collisions. Xin et al. [18] propose a time-space model and use a dedicated algorithm to minimize the cycle time of operation tasks for each AGV while avoiding collisions. Zone control is a simple and effective method to avoid collisions and deadlocks, which is widely used in AGV systems [19]. The guide path is composed of a series of zones that represent workstations, intersections of several lanes, or simple parts of a straight lane. Vehicles access to any zone must be authorized in advance by the central controller to avoid collisions and deadlocks [20], [21]. In [4], a chain of reservation (COR) based coordination method is proposed. In the method, the environment is divided into squares.
The path points of AGVs are reserved at the beginning of the task to avoid deadlocks and collisions. Ho et al. [22] propose a dynamic zone strategy which based on two procedures to prevent collisions and maintain the load balance between the vehicles of different zones. Ho et al. [23] propose a new zone control strategy which uses zone partition design and dynamic zone control method to prevent collisions. However, the strategy can only be used in a system with single-loop guide paths. Li et al. [24] adopt a strategy to find k-th shortest routes, from which a collision-and-deadlock-free route can be selected to improve the system efficiency. In [25], two effective traffic-control policies with polynomial-time complexity are proposed to avoid collisions and deadlocks. The authors exploit the authorization-based point control policy to avoid collisions. And the deadlock recovery policy is to eliminate the cycle in the full graph by re-planning one of the first AGVs that causes a deadlock. The design of safety zones is the key of the zone control. Too large or too small security zone division can affect the efficiency of collision and deadlock resolutions. Petri net, as an effective tool for modeling and analyzing deadlock problems, are often used in AGVs control systems [26]- [28]. In [29], the authors use Petri nets to decouple the upper level planning from the lower level logical control in an AGV system. In [30], the authors use time Petri net decomposition method to conduct AGV scheduling and conflict-free path planning. Uzam [31] propose an optimal deadlock-prevention policy which is based on the division of reachable states. However, the obtained policy is not structurally optimal. Wu et al. [32] present a deadlock-prevention controller based on resource-oriented Petri net (ROPN) model and zone control strategy. By taking advantage of ROPN, the relationship between bad markings and structural properties is revealed. In [33], an ordinary Petri-net (PN) based approach is proposed to design a programmable logical controller (PLC) for collision prevention. The authors use an ordinary PN to model the AGV system and its controllers. Then, the method to automatically translate a closed-loop PN into a ladder diagram program is proposed. Petri net, as a powerful tool for discrete-event systems (DESs) analysis and synthesis, is often used in investigations of AGV control system properties [15]. However, Petri net-based methods can cause state explosions since at each time a robot needs to check the whole state space to determine whether it is safe to move back to its current state. As mentioned above, many different control policies are proposed to realize collision-and-deadlock-free travelling in transport systems with multi-AGVs. However, most of the existing approaches are less efficient for collision and deadlock avoidance in dynamic environments.
The aim of this paper is to improve the efficiency of the collision and deadlock resolution in dynamic environments. In this work, a dynamic resource reservation based (DRR) method to prevent collisions and deadlocks is proposed. The shared resource points of each AGV are calculated in real time. We exploit two conditions to make decisions for dynamical resource reservations. The main novelty of this paper lies VOLUME 8, 2020 in dynamic reservations of resource points, which can ensure AGVs' collision-and-deadlock-free travelling while improving the time efficiency. The control algorithm is implemented on both central and local controllers.
The rest of the paper is structured as follows. Section 2 presents the system description. Section 3 introduces the main design of our proposed method. In section 4, the simulation results are presented. Section 5 draws the conclusions.

II. THE ENVIRONMENT DESCRIPTION
An industrial application often involves a number of AGVs delivering goods and materials among workstations and storage pipes. We consider R AGVs that are moving in the same environment, which can be denoted as R r = {r r : 1 ≤ r ≤ R}. The environment is completely known and structured.
For ease of modeling, the layout is divided into square blocks with the same size, as shown in Fig. 1. Yellow blocks represent the parking places. Green blocks represent the passable guide path blocks. Each passable path block is assigned to specific traveling directions represented as arrows in the block. Vehicles can stay or rotate at the defined areas. They are positioned at the center of these fields. From this point of view, the traveling of AGV can be described as moving from one block to the next block. Note that AGVs cannot move to the next blocks during the deceleration time, which can be realized by designing the AGV speed and the block size reasonably in reality. Moreover, considering the mathematical analysis and supervisory control algorithm design, the layout of a multi-AGVs transport system can be further abstracted into an undirected graph G = (N , A). The blocks are abstracted as points and the adjacent points are connected by virtual directed arcs, as shown in Fig. 2. The green points represent the center of passable guide path blocks. The yellow points represent the center of parking places.
The set of all the points in the graph is denoted as N = Each vehicle has its own parking place. The parking places are the origins of motions and all AGVs wait for a transport task at their parking places. The set of parking places is described as It means that the number of AGVs can be equal or less than the number of parking places. And there exists at least one point that is not a parking place. The resource points represent passable guide path points, which can be denoted as Note that the parking place of each AGV cannot be a resource point. Each AGV can move from parking places to resource points and vice versa. The set of active tasks is represented as M a = {m i : m i ∈ M }, where M is a set of all tasks generated by the transport system. Assumption 1: Each parking place n x ∈ N o is adjacent to at least one resource point n y ∈ N s . Assumption 2: All AGVs in the system cannot change their transport routes.
In most of the coordinate algorithms, the transport routes of AGVs are assumed unchangeable [2] [12]. Because the path re-planning of AGVs may lead to new collisions and deadlocks. In a sense, re-planning does not resolve the collisions and deadlocks completely. And it does increase the computing complexity of the system.

III. DRR-BASED COLLISION AND DEADLOCK PREVENTION
In this section, a collision and deadlock prevention method is proposed to eliminate collisions and deadlocks in the transport system. First, the calculation of shared resource points is introduced. Then, we elaborate the AGVs how to reserve the resource points according to two conditions. Lastly, the implementation of two control policies on both central and local controllers is described in detail.

A. SHARED RESOURCE CALCULATION
AGVs move along the guiding paths and occupy spaces. During the movement, AGVs often collide with each other because of the limited capacity of the area. When a task is assigned to an AGV, it reserves, occupies, and releases the resource points. Each AGV can occupy only one resource point at a time.
Definition 1: The state of AGV r a in the system can be idle (s a = 0), moving (s a = 1) and waiting (s a = 2). s a = 0 indicates that r a have no task to execute. The moving state s a = 1 indicates that r a is on the move. s a = 2 indicates that r a is waiting for the desired resource points that have been reserved by another AGV.
Definition 2: The state of the resource point n x in the system could be idle (s(n x ) = 0), occupied by AGV r a (s(n x ) = r a ).
s(n x ) = 0 indicates that there is no AGV reserving n x . s(n x ) = r a indicates that n x has been reserved by AGV r a and the others cannot pass by n x .
The tasks that AGVs have to accomplish correspond to transporting an item from a pick-up point to a target point. Thus, once a task is assigned to AGV r a , the path p a that the vehicle has to follow is computed. A path is simply a sequence of adjacent points, which can be denoted as p a = {n x , n x+1 , · · · }.
In the transport system, AGVs can send their real-time information, including their locations, to the central controller. From this information, we can determine the state with respect to the motion progress of AGVs, i.e., which points are occupied by the AGVs currently, and which ones are the next desired points to be occupied in the next step.
Definition 3: The travelling information of an active AGV r a is described as I a = {n x , n x+1 }, where n x is the current point, and n x+1 is the next desired point.
Definition 4: The residual transport route a of AGV r a represents the residual sequence of points that remain to be visited by r a before finishing the task.
Definition 5: For an active AGV r a , a consists of an ordered sequence of resource points shared with other AGVs, which can be denoted as a = {n x : n x ∈ a , n x ∈ b , b = a}, where b is the number of another AGV.
When AGVs share resource points with some other AGVs, collisions and deadlocks may occur. In order to solve this problem dynamically, it is necessary to record of each AGV in real time. After AGVs reach the desired points, is updated. At the same time, of each AGV is updated according to . The procedure is described in detail in Example 1.
Example 1: Shared resource points calculation as shown in Fig. 3.

B. DYNAMIC RESOURCE RESERVATION
Obviously, there are two types of collisions in the system, as shown in Fig. 4.
Definition 6: Collision: An AGV reserves or occupies a resource point that belongs to another AGV, and still moves to that point.
Pursue collision occurs when an AGV strike another one on the rear. Cross collision occurs when two AGVs are occupying the same resource point at the same time.
There are two types of deadlocks as shown in Fig. 5. Definition 7: Deadlock: The related AGVs cannot move on because each of them has to occupy the points occupied by VOLUME 8, 2020 another AGV. A deadlock occurs when two AGVs reserve the shared resource at the same time.
It is worth noting that the deadlock is actually an unresolvable collision.
Heading-on deadlock ( Fig. 5(a)) occurs when two AGVs travelling at adjacent points at the same time, but the directions are opposite [13]. Loop deadlock (Fig. 5(b)) is a situation where the travelling points of the related AGVs form a closed loop.
The method proposed in this paper for collisions and deadlocks prevention is realized by changing the motion states of AGVs according to the states of the resource points. Two conditions are set to judge whether the point can be reserved by the AGV r a . Only if the AGV satisfies all the required conditions can it go to the next desired point. Otherwise, the AGV would stop at the current resource point.
Condition 1: n x+1 / ∈ a and s(n x+1 ) = 0. It means that the next resource point of r a does not belong to the sets of guide paths of any other AGVs.
Condition 2: n x+1 ∈ a , but ∀n y ∈ a , s(n y ) =r b , where r b is another active AGV.
It means that the next resource point of r a belongs to the guide paths of other AGVs, but all the shared resource points in a are not reserved by the other AGVs.
Summing up, when the next desired point of AGV r a satisfies condition1 or 2, the next desired point n x+1 can be reserved by it. It means that the state of point n x+1 is s(n x+1 ) = r a . Otherwise r a stops. After departing from the current point n x , the state of point n x changes from s(n x ) = r a to s(n x ) = 0. The AGV repeats this operation along the whole route.
Theorem 1: ∀r a ∈ R r : the movement of r a based on DRR is collision-free in the system.
Proof: As mentioned above, r a can start moving to the next point if and only if the next point can satisfy condition 1 or condition 2, which means the next point is only reserved by r a . After movement, r a removes the occupation of the current point. For condition 1, the next point does not belong to the shared resource points of r a . Thus, there are obviously no collisions. For condition 2, the shared resource points can only be reserved by one AGV at a time, i.e. two AGVs cannot move to the same point at the same time. Therefore, collisions between AGVs are impossible.
Theorem 2: ∀r a ∈ R r : the movement of r a based on DRR is deadlock-free in the system.
Proof: According to theorem 1, collisions between AGVs are impossible, which means the shared resource points cannot be reserved by two AGVs. Based on the definition of the deadlock, deadlocks between AGVs are impossible.
Example 2: Heading-on deadlock prevention as shown in Fig. 6.
There are three unfinished tasks that performed by r 1 , r 2 and r 3 . The corresponding residual routes can be represented as 1 ={7,6,5,4,3,10,11}, 2 ={3,4,5,6,7,8}, 3 ={13,7,14,15}. The sets of shared resource points are Because the next point 7 of r 1 belongs to 1 , according to condition 2, there are no points in 1 occupied by other AGVs. The point 7 can be reserved by r 1 .r 1 removes the occupation of point 8 and moves on. As for r 2 , the next point 3 belongs to 2 , but the shared resource points have been occupied by r 1 , so r 2 will change its state from moving to waiting and cannot remove the occupation of point 2. The next desired point 13 of AGV r 3 is not a shared resource point, and thus r 3 can move on. Therefore, deadlocks between AGVs are impossible.
Example 3: Loop deadlock prevention as shown in Fig. 7. There are four AGVs in the system that can be represented as R r = {r 1 , r 2 , r 3 , r 4 }. The corresponding residual routes can be represented as 1 16}. We assume that AGVs receive the task according to the AGV number. When r 1 arrives at point 17, r 2 , r 3 , r 4 arrive at 9, 20, 28 in turn. The travelling information is I 1 = {17, 16}, I 2 = {9, 15}, I 3 = {20, 21} and I 4 = {28, 22}. Because the next point 16 of r 1 belongs to 1 , according to condition 2, there are no points in 1 occupied by other AGVs. The point 7 can be reserved by r 1 . In the same way, r 2 and r 3 can reserve the points 15 and 16, respectively. As for r 4 , the next point 22 belongs to 4 and 3 . However the shared resource point 16 has been occupied by r 1 , so r 4 will change its state from moving to waiting and cannot remove the occupation of the point 28. Therefore, deadlocks between AGVs are impossible.

C. ALGORITHM IMPLEMENTATION
The above-mentioned AGV collision and deadlock prevention method can be applied to the traffic system control. In this section, we elaborate the control algorithm implemented on both central and local controllers. The control policy of the central controller is described in detail in Algorithm 1. In this algorithm, the set M is a task list with specified order. R r is a set of idle AGVs.
The term s (n x+1 ) = r i means that the next desired point n x+1 is reserved or occupied by r i according to the conditions mentioned above. At the beginning of the simulation, the central controller generates path information and sends the information to an idle AGV (line2-8). When the central controller receives the travelling information that indicates AGVs arrive at a point, the sets i and i are updated (line 10-18). The arrival of the AGV r i at the parking point of the task indicates that task m i has been completed. Then, the completed task m i is removed from the task list M . Above procedures are repeated by the central controller until all the tasks in M are completed. The algorithm implemented on the local controller is described in Algorithm 2. When vehicle r i receives the task, the travelling information I i = {n x , n x+1 } is generated according to the current position and the route p i . As mentioned above, r i can move to the next point if and only if the point is reserved by it, i.e. s (n x+1 ) = r i . Once r i starts moving, the central controller removes the occupation of n x . According to the property of the point, AGV can pick up, unload, stop or move on to the next point. The AGV r i repeats this operation until the task m i is finished (line 5-20). After the AGV finishes the task, its state becomes idle immediately, and then it can receive a new task.
As mentioned above, the computational time of our algorithm mainly depends on the calculation of the shared resource points. For calculating the shared resource point set of an AGV, the central controller compares its residual path with those of the other AGVs. Thus, the computation time is positively related to the number of AGVs, i.e., the computational complexity O(N ).

IV. SIMULATION
To the best of our knowledge, COR-based collision and deadlock prevention method is recently updated and stateof-the-art method used in scenarios similar to this paper [4]. Therefore, to demonstrate the effectiveness of the proposed method, extensive simulations have been performed and the results are compared with COR. For this purpose, we first present a performance overview of DRR with different number of AGVs. Then the impact of the number of tasks and dynamic obstacles on the efficiency of collision and deadlock prevention is shown. Lastly, we provide an insight into the mechanism of collision and deadlock avoidance via recording the procedures of AGVs.

A. SIMULATION SETTINGS
The experiment layout with bidirectional lanes is shown in Fig. 8. The size of the square block is 1m ×1m and the size of the AGVs is 0.7m×0.7m. In the layout, we have 72 stations which are colored in blue. Each station can be a pick-up or delivery point. There are 12 parking points marked in yellow. Guide paths are represented as green block squares. Unless otherwise specified, the velocity of the AGVs is set as 1m/s and the turning time is set as 1s.
In this paper, we focus on the collision and deadlock prevention in the transport system. In the experiment, the tasks that delivery goods and materials among workstations are generated in a task list. The central controller assigns the tasks to specified AGVs according to the order in the task list. The route of each AGV is initially generated by using the A * algorithm.

B. PERFORMANCE METRICS
During the simulation, we use three main metrics to evaluate the effectiveness of the collision and deadlock prevention method proposed in this paper.
• Total travelling time T : T = T e − T s , where T s is the starting time of the first task and T e is the time all the AGVs stop at the parking places.
• Waiting time of each AGV T w : is the time of AGV has to stop to avoid deadlocks and collisions for performing task m i and m is the number of all the tasks generated in the transport system.
• Average waiting time T a : the average of the waiting time of all AGVs.

1) PERFORMANCE OVERVIEW
In this simulation, we compare the total travelling time of DRR and COR. The number of tasks is set as 120. The number of vehicles varies from1 to 12 and the rest of the parameters remain the same as the default values in Section IV-A. The relation between the number of AGVs and the total travelling time is shown in Fig. 9. The total travelling time of DRR is 11%∼28% less than that of COR with different number of AGVs while having almost equal travelling distance. This is because DRR reserves the resource points dynamically compared to COR in which the resource points are reserved at the beginning of the tasks. From Fig. 10, it can be seen that the average waiting time of each AGV of DRR is 25%∼50% less than that of COR. As the number of AGVs increases gradually, the total travelling time of the two methods decreases. The best solution of DRR and COR is found with 9 and 7 vehicles, respectively. However, further increase of the number of vehicles does not bring a significant improvement, probably due to the rapidly increased congestions. Overall, with different number    of AGVs, DRR proposed in this paper is more efficient than COR.
In practice, there are usually dozens of AGVs working in the environments. In order to verify the performance of DRR with large-scale AGV systems, we further implement DRR in a 400m 2 topological graph. The number of tasks is set as 120. The number of vehicles varies from10 to 80. The relation between the number of AGVs and the total travelling   time is shown in Fig.11. The total travelling time of DRR is 4.6%∼28.8% less than that of COR with different number of AGVs. From Fig. 12, it can be seen that the average waiting time of DRR is 9.2%∼46.2% less than that of COR.

2) IMPACT OF THE NUMBER OF TASKS
In this simulation, the impact of the number of tasks on the performance of those two methods is investigated. We set the number of vehicles as 6. From Fig. 13, we can see that the total travelling time with different number of tasks based on DRR is 22%∼40% less than that of COR. However, the travelling distance of both methods is almost equal, as shown in Fig. 14. As expected, from Fig. 13, we can also see that the total travelling time and the total travelling distance increase for both methods as the number of tasks increases. For different number of tasks, the DRR shows better performance with less total travelling time.

3) IMPACT OF DYNAMIC OBSTACLES
In practice, external obstacles (e.g. people, manual forklifts) often appear in the working environment of AGVs. In order to verify the performance of DRR with dynamic obstacles, we implement DRR in a 5 * 5 topological graph with 12 AGVs. The velocity of each AGV is set as 0.5m/s.
In the simulation, we add dynamic obstacles with a residence time of 10s to random position. From Fig. 15, it can be seen that there are no collisions and deadlocks in such a dynamic and congested situation (one AGV per 2 square meters). And the total travelling time increases as the number of the obstacles increase.

4) PROCESS OF LOOP DEADLOCK PREVENTION
To have an insight into the detailed mechanism of collision and deadlock prevention method, we implement DRR in a 5 * 5 topological graph with 4 AGVs. As shown in Fig.16, we assume that four AGVs are assigned tasks according to the ID of each AGV. The routes of AGVs are represented as

V. CONCLUSION
In this paper, a dynamic resource point reservation based collision and deadlock prevention method DRR is proposed, which can ensure collision-free AGVs travelling while achieving high time efficiency. DRR changes AGV motion states to prevent collisions and deadlocks dynamically, depending on whether the points have been reserved. Simulation results show that the total travelling time of DRR is 11% ∼ 28% less than that of COR with different number of AGVs while having almost equal travelling distance. The total travelling time and waiting time of each AGV of DRR are less than those of COR with different number of tasks. The future work should focus on optimizing the task dispatch sequence that can reduce the AGV task execution time while ensuring collision-and-deadlock-free travelling.